Keepfor.me

Keepfor.me

[Release] [Tests] [Security] [Runtime] [FastAPI] [Protocol] [License] [Support this project]

A thin, fast read-it-later and personal library application running entirely on Cloudflare Python Workers, D1 SQLite with FTS5, Vectorize, R2, and the Model Context Protocol (MCP).

Features

• Four Core Jobs: Capture, Store, Search, and Share with AI Agents.
• Python Edge Runtime: Powered by FastAPI on Cloudflare Python Workers (Pyodide).
• Installable PWA & Native Sharing: Add to Home screen on Android and iOS. On Android, native Web Share Target integrates directly into the system share sheet. On iOS, 1-tap Apple Shortcuts integrate with the system share sheet using your Safari session, paired with smart clipboard detection in the PWA.
• Distraction-Free Reader: Customizable themes (Light, Sepia, Dark), fonts (Sans, Serif, Mono), and font sizes.
• Hybrid Search: Reciprocal Rank Fusion (RRF) combining D1 FTS5 BM25 keyword matching and Vectorize semantic embeddings (bge-base-en-v1.5). FTS and embedding calls run concurrently, and the vector path is time-bounded so a slow AI binding degrades to keyword-only instead of hanging.
• Resilient Content Extraction: Automated body parsing via trafilatura with OpenGraph metadata fallback so bookmarks are never lost. Browser-identical request headers plus an optional reader-proxy fallback (configurable via READER_PROXY_BASE, off by default) recover many bot-walled (HTTP 403) origins.
• Visible Extraction State: Every item shows Extracting…, content, or a failed reason (e.g. HTTP 403) rather than silently stalling. Bulk imports fan out through the queue, never blocking the request.
• Dual Snapshots in R2: Raw original HTML snapshot (raw.html) and sanitized reader HTML (clean.html).
• Trash, Not Data Loss: Deleting (from the UI or from an agent) moves an item to the Trash instead of destroying it. The row, the snapshots and the vectors stay, so Restore is instant; Delete forever in the Trash view is the only permanent path, and a scheduled purge clears the Trash after 30 days.
• Zero-Config Single-Tenant Lock: First registration automatically claims admin ownership and locks out external signups.
• Model Context Protocol (MCP): Remote Streamable HTTP server at /api/mcp (stateless, Bearer PAT auth) for opencode, Claude Desktop, and Cursor — no local install needed.
• Instant Browser Capture: Drag-and-drop JavaScript bookmarklet and unpacked Manifest V3 extension.
• Import & Export: Ingest Pocket, Instapaper, Omnivore, and browser bookmark exports (Netscape HTML & CSV). Chrome folder names are preserved as tags. Notes import from Google Keep Takeout zips, Evernote .enex files, and zips of Markdown/text files (Obsidian vaults, Apple Notes via Exporter, Simplenote): tags and original dates are kept, links inside notes become regular saves, attachments are listed but not stored, and re-imports never duplicate (50 MB cap for zips and .enex, 10 MB for HTML/CSV; inside a zip: 2 MB per file, 200 MB total and 20,000 files, so decompression bombs are refused). Every notes upload keeps a status record with its progress and final counts (notes imported, duplicates skipped, links queued, attachments not imported, parse errors, plan-limit skips) and a capped list of error messages, so the result survives a refresh.
• Mobile-First UI: Bottom tab bar and tag chip strip on phones, ≥32px touch targets, no horizontal overflow down to 360px.

Architecture Overview

flowchart LR    Browser["Web UI / PWA / Share Sheet"] -->|FastAPI| Worker["Cloudflare Python Worker"]    Claude["Claude Desktop / MCP Agents"] -->|Streamable HTTP + Bearer PAT| Worker
    Worker -->|Queries & FTS5| D1[("D1 Database")]    Worker -->|Enqueue job| Queue["Cloudflare Queue"]    Queue -->|Process batch| Worker    Worker -->|Raw & Clean HTML| R2[("R2 Storage")]    Worker -->|Embeddings| WAI["Workers AI"]    Worker -->|Upsert vectors| Vec[("Vectorize")]

Extraction lifecycle

Saving a URL inserts the row with status='queued' and enqueues a job; the queue consumer fetches, extracts, stores snapshots, and flips it to ok or failed with a reason. Failures are always visible in the UI and reader.

Two deliberate design choices:

• No queue binding → extract inline. Local dev and tests have no queue consumer, so save_item extracts synchronously rather than leaving items stuck in queued forever.
• Bulk imports never run inline. A 900-bookmark import would issue thousands of D1 writes inside one request; instead it fans out as queue messages of 25 bookmarks, or a background task when no queue is bound.

If messages are ever lost, Settings → Re-queue stuck items re-sends jobs for rows stuck in queued (skips fresh rows so live messages aren't duplicated).

Trash and restore

Deleting never destroys. MCP delete_item and DELETE /api/items/{id} set items.deleted_at and stop there: the row, the R2 snapshots and the vector embeddings stay, so an agent cannot destroy library data. Trashed items are invisible everywhere (library, search, MCP reads, counts, exports) until they are restored.

The Trash view (/trash) lists them with two actions: Restore brings an item back into the library, Delete forever is the only permanent delete. Hosts can also run the scheduled cleanup themselves:

from keepfor.models.items import purge_trash
# Hard-deletes trashed items older than 30 days (row, snapshots, vectors).await purge_trash(db, env, user_id=None, older_than_days=30)
# A multi-tenant host passes the tenant scope: it carries the R2 prefix# and the Vectorize namespace of the tenant. An unscoped call is refused.await purge_trash(db, env, user_id=None, older_than_days=30, scope=scope)

Nothing is deleted behind the user's back: without a scheduler the Trash just grows.

Reader Proxy Fallback (READER_PROXY_BASE)

By default, Keepfor.me does not route failed fetches to any third-party service. When an origin rejects a direct fetch (HTTP 403, 429, 530), the item is recorded as failed.

To opt in to reader proxy fallback (e.g. using Jina Reader or a self-hosted readability proxy):

• Configure READER_PROXY_BASE in wrangler.jsonc (vars.READER_PROXY_BASE = "https://r.jina.ai/") or in .dev.vars.
• Privacy Notice: When enabled, blocked article URLs will be transmitted to the configured proxy service. Private IP ranges (e.g., 127.0.0.1, 10.0.0.0/8, 192.168.0.0/16) and local domains are strictly blocked by SSRF egress policy and are never forwarded to any reader proxy.

For full guidance on private / offline deployments and egress policies, see docs/self-hosting.md.

Deployment

1. Prerequisites

• Node.js 20 or newer and npm
• uv 0.12.3 or newer
• Cloudflare account with Workers, D1, R2, Vectorize, and Queues enabled
• Custom domain: keepfor.me (or standard *.workers.dev subdomain)

2. Provision Cloudflare Resources

Run the following commands once to create your D1 database, R2 bucket, Vectorize index, and Queue:

# 1. Create D1 Databasenpx wrangler d1 create keepfor-me-db# Copy the returned database_id into wrangler.jsonc
# 2. Create R2 Bucketnpx wrangler r2 bucket create keepfor-me-bucket
# 3. Create Vectorize Index (768 dimensions for bge-base-en-v1.5)npx wrangler vectorize create keepfor-me-index --dimensions=768 --metric=cosine
# 4. Create Queuenpx wrangler queues create keepfor-me-queue

3. Apply Database Migrations

Apply the initial schema (tables, FTS5 virtual table, indexes):

npx wrangler d1 migrations apply keepfor-me-db --remote

For local testing:

npx wrangler d1 migrations apply keepfor-me-db --local

4. Deploy

Deploys are manual: self-hosters deploy to their own Cloudflare account via pywrangler deploy or GitHub Actions workflow_dispatch (with input confirm: "deploy"). Pushing to main does not auto-deploy.

[!WARNING] wrangler.jsonc has "remote": true on D1, R2, and Vectorize bindings. Never run local dev against wrangler.jsonc as it interacts directly with remote resources. For local simulation, copy wrangler.local.example.jsonc to wrangler.local.jsonc and run uvx --from workers-py pywrangler dev --config wrangler.local.jsonc.

PyWrangler bundles the Python dependencies declared in pyproject.toml.

# Always dry-run first and check the "Total (N modules)" row: the deploy# hard-fails if the bundle exceeds 64 MiB.uvx --from workers-py pywrangler deploy --dry-run
uvx --from workers-py pywrangler deploy

A healthy bundle is roughly 52,000 KiB / ~4,500 modules. If .venv-workers/ paths appear in the module table, the build venv is being bundled and the deploy is one dependency bump away from failing.

Once deployed, visit your domain (e.g., https://app.keepfor.me or https://keepfor-me.<your-subdomain>.workers.dev). The first user to register automatically becomes the admin and locks public registration.

5. Custom Domain

wrangler.jsonc declares a route for app.keepfor.me. Create the DNS record once (Cloudflare dashboard → DNS → Add record):

The Worker route matches before the IP is ever contacted, so the placeholder address is fine. workers_dev is left enabled so the *.workers.dev URL keeps working.

Connecting AI Agents (Remote MCP)

Keepfor.me hosts a remote Streamable HTTP MCP server at https://app.keepfor.me/api/mcp, exposing 42 tools for capturing, finding, reading and tagging your library:

1. save_url: Save any webpage to your library.
2. save_urls: Save many webpages to your library in one call, with a status for each URL.
3. save_note: Save a note, snippet or prompt template.
4. search_library: Hybrid search across text and meaning.
5. search_passages: Passage-level search with permanent snapshot URLs.
6. find_item: Look up an item id by title or url (no search, no embeddings).
7. get_item: Read metadata and excerpt by id, url or exact title.
8. read_item: Paginated full-text reading.
9. list_items: Compact paginated browse with sorting and filters.
10. reading_history: What you saved and opened, with date filters.
11. related_items: Items similar to a given item or text.
12. list_tags / create_tag / describe_tag / update_tag / rename_tag / merge_tags / delete_tag: Full tag management.
13. tag_item / tag_items / set_item_tags / retag_by_query: Tagging, in bulk and safely (dry-run by default).
14. bulk_update: Update status, pin and tags on every item matching a filter (dry-run by default).
15. suggest_tags / search_tags / tag_stats / related_tags / tag_cleanup: Tag vocabulary tools.
16. find_cleanup: Read-only cleanup report (duplicate urls, dead links, never-opened old items, untagged items, near-duplicate tags).
17. delete_item / pin_item: Remove or pin an item.
18. library_digest: A compact recap — saves, opens, top tags, unread highlights.
19. create_highlight / list_highlights / delete_highlight: Quote a passage of an item, anchored by character offsets.
20. create_collection / list_collections / add_to_collection / remove_from_collection / rename_collection / delete_collection: Named buckets of library items.
21. get_account: Your plan, the end of the trial and the quota left — free, so it still answers when the quota is exhausted.

The server is stateless (no sessions, plain JSON responses, no SSE stream) and authenticates with the same Personal Access Tokens as the REST API.

Setup

1. Go to Settings & API in your Keepfor.me UI.
2. Click Generate Token to create a Personal Access Token (kfm_live_...). Pick Read-only in the scope picker if the agent should only be able to browse and search the library.
3. Point your client at the endpoint with the token as a Bearer header:

opencode (opencode.json):

{  "mcp": {    "keepfor-me": {      "type": "remote",      "url": "https://app.keepfor.me/api/mcp",      "headers": { "Authorization": "Bearer {env:KEEPFOR_ME_TOKEN}" }    }  }}

Then export KEEPFOR_ME_TOKEN=kfm_live_... before launching opencode, and restart opencode after changing the config.

Cursor / Claude Desktop: add a custom remote MCP server with URL https://app.keepfor.me/api/mcp and header Authorization: Bearer kfm_live_YOUR_TOKEN_HERE.

Handshake check without any client:

curl https://app.keepfor.me/api/mcp \  -H 'Content-Type: application/json' \  -H 'Accept: application/json, text/event-stream' \  -H "Authorization: Bearer kfm_live_YOUR_TOKEN_HERE" \  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Available MCP Tools (Server 0.2.0)

The MCP server exposes 42 tools. The tools that change many items at once take a dry_run flag: it is only accepted as a JSON boolean (omitted means true, a preview), and anything else is rejected rather than applied.

1. save_url — Save any webpage to your library. Accepts optional agent_summary, why_saved, source_context, saved_by for agent research provenance.
2. save_urls — Save up to 50 webpages to your library in one call, with a status per URL (queued, exists, invalid, over_limit) and the totals. Needs the write scope. The URLs are deduped by canonical URL, every new URL goes through the same save check as save_url (the ones the save limit refuses come back as over_limit), and the extraction is queued, so the call does not wait for the pages.
3. save_note — Save a personal note, code snippet, or prompt template. Accepts the same optional provenance fields.
4. search_library — Hybrid search across text and meaning (item-level results with snippets).
5. search_passages — Passage-level hybrid search over semantic chunks with max_chars budget. Returns chunk_id, char offsets, and permanent snapshot_url (R2 archive) that survives link rot.
6. read_item — Token-efficient paginated reading of full content. Returns text, next_cursor, total_chunks, outline.
7. get_item — Retrieve metadata and excerpt, addressed by item_id, url or exact title. By default caps content_text to excerpt and reports truncated and content_length; use include_content=true for full text.
8. list_items — Compact paginated browse (id, title, url, saved_at, tags — no body). Sorts by saved, opened or title; filters by since/until, domain, pinned, type and tags.
9. find_item — Find item ids by exact/prefix title or exact/normalized url. No search, no embeddings: the cheap path from a title you know to an item_id you can read.
10. reading_history — Reading history with date filtering (since/until), event types (saved/opened), and optional query/tag filters.
11. related_items — Nearest neighbors by vector similarity from an item_id or raw text, with one-line reasons.
12. list_tags — Every tag with its item count and last-used date. Sorts by count, name or recent; filters by prefix and min_count.
13. create_tag — Create a tag with optional description, color and parent (hierarchy, e.g. work/research).
14. describe_tag — A tag's metadata, parent, children and item count.
15. update_tag — Edit a tag's description, color or parent.
16. rename_tag — Rename a tag; every item carrying it is updated.
17. merge_tags — Fold several tags into one target tag (at most 20 source tags per call).
18. delete_tag — Remove a tag from items. Items are never deleted. dry_run=true by default: returns the affected count and a sample; dry_run=false applies.
19. tag_item — Add or remove tags on a single item.
20. tag_items — Add/remove tags across many items. dry_run=true by default. At most 100 items per call; the result reports total_matched and truncated.
21. retag_by_query — Tag items matching a title/url query (plus tag filters). dry_run=true by default. At most 100 items per call; the result reports total_matched and truncated.
22. bulk_update — Update the status, the pin state and the tags of every item matching a filter (status group, saved date range, domain, pinned, type, tags). dry_run=true by default: it reports total_matched, affected and truncated without writing anything. At most 200 items are updated per call, and an empty filter or an empty change is rejected. The whole update is applied in one transaction, and tag names follow the create_tag rules (2-64 characters of a-z, 0-9, space, _ or -).
23. set_item_tags — Replace the whole tag set of one item.
24. suggest_tags — Suggest tags for an item or for free text, drawn from your existing vocabulary first.
25. search_tags — Fuzzy search over tag names and descriptions (machinelearning finds machine-learning).
26. tag_stats — Tag usage statistics for the library.
27. related_tags — Tags that co-occur with a tag on the same items.
28. tag_cleanup — Near-duplicate tags, unused tags and untagged items worth a look.
29. find_cleanup — Read-only cleanup report: duplicate items (same normalized url), dead links (recorded extraction failures — no network requests), never-opened old items, untagged items and near-duplicate tags. Takes categories, older_than_days and a per-category limit (max 100); each category reports the candidates, a count and a truncated flag. The item categories list one id with a reason; the group categories list one entry per group — duplicates gives the group's ids (capped at 10), its size and whether the id list was truncated, near_duplicate_tags gives the group's tag names — so limit caps groups and one url saved 300 times stays a small response.
30. delete_item — Move an item to the Trash. The item leaves the library but is not destroyed: it stays restorable for 30 days, and only the user can delete it for good from the Trash view.
31. pin_item — Pin or unpin an item to/from the top shelf.
32. library_digest — Read-only recap of the library for a period: saves, opens, top tags and unread highlights. Takes optional since, until and tag; every list in the response is capped.
33. create_highlight — Quote a passage of an item and anchor it with the character offsets it covers inside content_text. start_offset defaults to 0, end_offset to start_offset + len(text); an end offset past the end of the item's content is refused.
34. list_highlights — The highlights of one item, ordered by their position in the text, with the quoted text, offsets, optional note and color.
35. delete_highlight — Remove one highlight. The item it belongs to is never deleted.
36. create_collection — Create a named bucket of library items, with an optional description. Creating one with an existing name returns it unchanged.
37. list_collections — Your collections with their item count and item ids (at most 20 per collection, with items_truncated when there are more).
38. add_to_collection — Put items in a collection, addressed by name or id. At most 100 items per call; the result reports added, skipped and truncated.
39. remove_from_collection — Take items out of a collection. The items themselves are never deleted.
40. rename_collection — Rename a collection. Renaming onto a name that is already taken is refused, never merged.
41. delete_collection — Delete a collection and its links. The items it held are never deleted; use remove_from_collection to empty it.
42. get_account — Read-only report of your account: the plan, trial_ends_at and the quota of every metered action (limit, used, remaining, resets_at). The call is free — it costs no quota unit, so it still answers when the quota is exhausted.

All tools declare MCP annotations (readOnlyHint, destructiveHint, idempotentHint). They are advisory hints for the client, and the server uses them to enforce the scopes of a token (see below). A token is never rejected for missing scope information: no scope information means full access. Each description says when to use a tool versus its siblings. List-type responses include limit, offset/cursor, and a fields projection; the default list_items page stays under 4 KB.

Tag filters (tags_all, tags_any, tags_none, untagged) work on list_items, search_library, search_passages, find_item, reading_history and bulk_update, and count child tags: filtering on work also matches work/research.

MCP Resources and Prompts

The server also speaks the resources and prompts parts of the MCP spec, with the same auth and tenant scoping as the tools.

Resources — read-only views of your own library:

resources/list returns your most recent items and most used tags, resources/templates/list returns the two URI templates, and resources/read returns the payload as JSON.

Prompts — reusable templates that read your own data:

Token scopes (read-only tokens)

A Personal Access Token can be limited to read-only. Create it on Settings & API with the Scope picker next to the token name (Full access is the default):

• * (all) — the default. Every tool works, exactly as before.
• read — over MCP the token only sees the read tools in tools/list, and any write or destructive tool (save_url, save_note, tag_item, create_tag, update_tag, rename_tag, merge_tags, delete_tag, tag_items, retag_by_query, bulk_update, set_item_tags, delete_item, pin_item, create_highlight, delete_highlight, create_collection, add_to_collection, remove_from_collection, rename_collection, delete_collection) is refused with a JSON-RPC error (error.data.type = insufficient_scope) before it runs — nothing is written.

Tokens created before scopes existed keep their full access: their stored scope is NULL and means "all". Scopes are enforced everywhere a token works: over MCP each tool needs the scope its annotations declare, and over REST a read-only token gets 403 on every write route. The exception is the read-only POST routes /api/search and /search: a search is a read, so a read-only token may use them. /api/mcp enforces scopes per tool. PATs are refused on /settings/* entirely, so a token can never mint a broader one.

Self-hosters: the scope column comes from migration 0012_pat_scopes.sql, so apply the migrations (step 3 above) before deploying.

Browser & Mobile Capture

1. Android (PWA + System Share Sheet)

1. Open your domain in Chrome and tap Add to Home screen (or Install app).
2. To save from anywhere, tap Share → Keepfor.me in any app — the native Web Share Target opens a prefilled save sheet, and swiping back returns you seamlessly to your app.

2. iPhone & iPad (Apple Shortcut & Smart Clipboard)

iOS WebKit sandboxes PWAs from the system share sheet. Keepfor.me provides two zero-friction capture methods for iOS:

1. 1-Tap Apple Shortcut (Share Sheet): Go to Settings & API → iPhone & iPad Quick Sharing and tap Install Keepfor.me Shortcut. This adds a native action to your iOS Share Sheet that sends URLs directly to Keepfor.me using your existing Safari login session (no API tokens or technical configuration required).
2. PWA Smart Clipboard: Add the app to your Home Screen from Safari (Share → Add to Home Screen). When you copy a link in any app and switch to the Keepfor.me PWA, a floating toast detects the clipboard URL and lets you save it with a single tap.

3. Drag-and-Drop Bookmarklet (desktop)

Go to Settings and drag the Keepfor.me button to your browser's bookmarks bar. Click it on any page to open a quick-save dialog.

Bookmarklets always show a generic globe icon. To get the app icon: bookmark any page on your domain, edit that bookmark, paste the copied code (Settings → Copy code) as its URL, and name it Keepfor.me.

4. Browser Extension (Manifest V3)

1. Open Chrome/Brave/Edge and navigate to chrome://extensions/.
2. Enable Developer mode (top right).
3. Click Load unpacked and select the keepfor.me/browser-extension folder.
4. Click the extension icon, enter your Worker URL and PAT token once in Settings (⚙️), then save any tab with Alt+S.

Local Development & Testing

Install the project and its test dependencies, then run the pytest suite:

python3 -m pip install -e . pytest pytest-asyncio pytest-cov httpxpython3 -m pytest tests/ -q

Run pytest from the repo root — keepfor resolves as a namespace package only when the root is on sys.path.

Lint and format exactly as CI does (bare ruff check . uses different rules and will pass on things CI rejects):

ruff check keepfor/ tests/ --select=E,W,F,I,Nruff format --check keepfor/ tests/

To run the local Worker preview with its Python dependencies:

uvx --from workers-py pywrangler dev

Local dev has no queue consumer, so saved links extract inline and show content immediately.

Performance notes

Cold starts dominate perceived latency on Python Workers: page loads sit around 250ms warm but can spike past 1.5s when an isolate boots.

• Import graph hygiene: Heavy article parsers (trafilatura, bs4, lxml) are imported lazily inside the extraction code. A regression test asserts they stay out of the request import path.
• Concurrent search & I/O: Hybrid search runs D1 FTS and Workers AI embedding lookups concurrently (asyncio.gather), bounded by a 5s timeout.
• D1 query optimization: Composite indexes ((user_id, created_at DESC) and (user_id, status, created_at DESC)) eliminate SQLite in-memory temporary B-tree filesorts (USE TEMP B-TREE FOR ORDER BY). Status counts, feed items, and tag aggregations are parallelized via asyncio.gather, and multi-statement tag/deletion mutations are batched via execute_batch to minimize RPC roundtrips.

Support this project

If you find Keepfor useful, you can support its development:

[Support this project]

Support Keepfor with a donation on Stripe (preset $5, or choose your amount).

Contributing

Contributions are welcome! Please read CONTRIBUTING.md for details on our development workflow, testing, and pull request process.

All contributors must sign our Contributor License Agreement (CLA). You retain copyright ownership of your work; the agreement grants us the license necessary to operate the hosted service and offer commercial licenses.

License

Keepfor.me is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0-only).

• Self-Hosting: You can self-host freely for personal or organizational use without restriction.
• Network Services: If you modify Keepfor.me and offer it as a service over a network, you must publish your modified source code under the AGPL-3.0.
• Commercial Licensing: Commercial licenses and alternative licensing agreements are available upon request to support proprietary, enterprise, or closed-source deployments. See LICENSING.md for details.
• Prior Releases: Previous releases prior to this license change remain under the MIT License for anyone who already has them.
• Trademark: The project name and marks are governed by TRADEMARK.md.