Spomory

Spomory

English | 中文

Spomory gives AI clients a memory that persists across sessions — and is shared between all of them. Tell one of them something once (a project detail, a preference, a fact about yourself) and any of them can recall it later, without you repeating yourself. It's an MCP server, so it works with any MCP-compatible client: Claude Desktop, Cursor, Codex CLI, and Doubao (豆包) have each been verified end-to-end with a real client session (see "What it can do" below); ModelScope, Coze, and 阿里云百炼 (Alibaba Cloud Bailian) are also known-compatible, connecting the same way any remote MCP client does (Streamable HTTP, or SSE for 百炼 specifically, which doesn't support Streamable HTTP yet).

It runs as an MCP server, either locally on your own machine (free, nothing leaves your computer) or as a hosted cloud service (free signup, memory follows you across devices) — every client above supports both. The rest of this README walks through the local path (that's what "Get started" below covers); the fastest way into the cloud path is the web app at spomory.yliuai.com/get-started/remote. Exact local/cloud configuration for each individual client lives in docs/mcp_quickstart.en.md (organized one section per client — Claude Desktop/Cursor/Codex CLI/Doubao — each covering both connection methods).

Curious what it actually does before installing anything? There's a no-signup demo at spomory.yliuai.com — paste in some text and see it turned into the structured facts that would then follow you across Claude, Cursor, Codex CLI, and every other client you connect, persisting across sessions instead of resetting each time.

Get started

Requires Python 3.11+.

1. Install. Or skip this step entirely — step 3's config can point straight at uvx, which fetches and runs it on demand, nothing to pre-install:

pip install "spomory[llm,embedding,mcp]"

2. Give it an LLM. Spomory uses an LLM to turn what you tell it into structured facts. Any OpenAI-compatible API works — OpenAI, DeepSeek, Qwen, etc.:

export LLM_API_KEY=sk-...export LLM_BASE_URL=https://api.deepseek.com   # optional, defaults to OpenAIexport LLM_MODEL=deepseek-chat                 # optional, defaults to gpt-4o-mini

3. Connect it to your client. Claude Desktop, Cursor, and Codex CLI each need a few lines added to a config file, pointing at the spomory-mcp command. The exact steps, a config file example for each client, and a real gotcha we hit (macOS blocking a venv that lives under ~/Documents) are in docs/mcp_quickstart.en.md.

Clients without a config file — Doubao (豆包), Coze, ModelScope — connect instead through a point-and-click "custom connector" form (URL + headers, or command + args + env for a local process); see the Doubao section in the same doc (each client — Claude Desktop, Cursor, Codex CLI, Doubao — now has its own section there, with STDIO/HTTP/OAuth grouped underneath it).

The one thing that trips people up: the config needs the absolute path to spomory-mcp (run which spomory-mcp to find it) — the client doesn't necessarily launch it with your shell's PATH set. This whole gotcha (and step 1) disappears if command is uvx instead — see the quickstart guide for the exact JSON.

That's it. Data lives locally in ~/.memory-core/ by default (override with MEMORY_CORE_DATA_DIR). A Dockerfile is also included, for MCP directories/hosts that deploy from a container image instead.

Two more things worth doing right after connecting: copy skill/spomory/SKILL.md into your client's skills directory so it actually calls these tools on its own instead of sitting there unused; and if you're already using Claude Code, Codex, or Cursor, run spomory-import-sessions --client claude-code (or codex/cursor) to seed the graph from real session history instead of starting from nothing. Both covered in the quickstart guide.

Want the LLM calls local too, instead of a cloud API? Point LLM_BASE_URL at any local OpenAI-compatible server — vLLM, Ollama, llama.cpp, or MLX all work — and set EMBEDDING_PROVIDER=openai_compatible to route embedding the same way instead of the local sentence-transformers model, dropping the torch dependency entirely. Details and per-engine examples in docs/mcp_quickstart.en.md.

What it can do

Once connected, six tools become available inside the client:

All six are verified working end-to-end against real Claude Desktop, Cursor, Codex CLI, and Doubao sessions, both local and remote — see docs/mcp_quickstart.en.md for what "verified" means for each client.

How it works, for the curious

You don't need any of this to use Spomory — it's here for people who want to know what's actually happening underneath.

• Pluggable LLM / embedding providers: defaults to any OpenAI-compatible API (including Chinese-market LLM providers) + local sentence-transformers (default bge-m3, bilingual Chinese/English).
• Dual-layer incremental knowledge graph: entities and relations are modeled as independent layers; new data is only extracted and merged in, never a full rebuild. Exact-match filler input ("thanks", "好的", "ok", ...) is skipped before it ever reaches the extraction LLM call, since it can't contain an extractable fact — relevant cost protection on any unauthenticated endpoint. Defaults to a local LocalGraphStore (networkx + SQLite); a PostgresGraphStore cloud implementation also exists, and both share the same behavioral contract test suite.
• HippoRAG 2-style retrieval: the query is matched directly against triples rather than only against entity nodes; the matched seed nodes are diffused via personalized PageRank for multi-hop association, then assembled into a natural-language context (with source timestamps, so "when did I mention X" is answerable). Ranking on top of that decays a relation's relevance the longer it's gone without being retrieved, and boosts it back up (log-dampened, so it can't dominate PPR rank) the more times the same fact has been restated — a passive signal alongside the active ADD/UPDATE/DELETE/NOOP decisions below.
• Memory management: an ADD/UPDATE/DELETE/NOOP action space, with a rule-based default policy (RuleBasedPolicy) and a full GRPO training pipeline (memory_manager/train_grpo.py, actually run and verified on a real GPU).
• Memory passport export + true delete: a JSON-LD style export format, physical deletion, and an audit log.
• Multimodal image verification: image captioning → reuses the text extraction pipeline → CLIP cross-checks candidate triples. Honestly positioned as "verification," not "native cross-modal extraction."
• Cloud skeleton: FastAPI user auth/API keys/quotas, a Stripe webhook billing scaffold (skeleton-level only, not production-deployed).

Measured results

Real runs against DeepSeek on 84 QA pairs from LoCoMo-10 (conv-26, first 150 turns) — not cherry-picked, and not competitive with the bigger players' published numbers yet:

A prior run (before a fix that folds dates into extracted predicates so "when" questions are answerable) scored lower on accuracy but higher on recall (62.0%) — the fix traded some retrieval recall for a real +14.3-point accuracy gain, and we went and found out exactly why instead of just reporting the accuracy number: the date-folding instruction sometimes misfires on content-free small talk ("Thanks!" → "thanked on 2023-07-03"), and those extra low-value triples crowd out relevant ones out of the fixed top-10 retrieval window. Full numbers, per-category breakdown, and the side-by-side extraction comparison that found this are in docs/benchmark_smoke_test.md.

LongMemEval (xiaowu0162/longmemeval-cleaned oracle variant, first 10 of 500 questions):

The limitations here matter as much as the numbers:

1. Only 10 questions, not the full 500 — each question ingests ~27 turns on average (~27 real extraction calls plus one generation and one judge call), and this environment's LLM API calls go through a proxy with real latency; the full dataset would take tens of hours. This is a real run, not a mock, but it's a small sample and shouldn't be read as generalizing to the full dataset.
2. All 10 happen to be temporal-reasoning type — the dataset also has a multi-session type; load_longmemeval(limit=10) takes the first 10 entries in file order with no stratified sampling, so this sample isn't representative of the dataset as a whole.
3. Recall@10 = 100% is largely an artifact of the oracle variant's design, not a strong retrieval claim — the oracle variant pre-filters each question's haystack down to only the relevant sessions (no distractor sessions), which is considerably easier than a real deployment's memory store (hundreds/thousands of unrelated turns). This isn't the same task as the full (non-oracle) LongMemEval benchmark and shouldn't be compared directly against numbers other products report on that harder variant.
4. Strict-match accuracy (30%) is far below LLM-judged accuracy (80%), consistent with the same pattern seen in the LoCoMo results — substring matching systematically undercounts answers that are correct but worded differently.

Raw data: benchmarks/results/longmemeval_oracle_subset.json; the run script is benchmarks/run_longmemeval_subset.py.

For contributors: building from source

Everything below is for people who want to hack on the internals, run the test suite, or reach dependency groups beyond what running the MCP server needs (GPU training, the cloud API skeleton, multimodal verification). If you just want to use Spomory, you don't need any of this — see "Get started" above.

Project layout

src/├── memory_core/│   ├── graph/            # entity/relation models, storage adapters (local SQLite / cloud Postgres), incremental writes│   ├── retrieval/        # query→triple matching, personalized PageRank, context assembly│   ├── memory_manager/   # action space, reward functions, GRPO training script, policy inference│   ├── multimodal/       # image captioning + CLIP verification│   ├── mcp_server/       # MCP Server (the distribution entry point)│   ├── export/           # memory passport export format + true delete│   ├── llm/              # pluggable LLM/embedding providers│   ├── audit.py          # deletion audit log│   └── usage.py          # retention/usage tracking└── cloud_api/            # FastAPI cloud service skeleton (auth, quotas, billing)benchmarks/                # LoCoMo/LongMemEval evaluation harness + multimodal comparison experimentstests/                     # 94+ tests, from unit tests to real LLM/GPU/Postgres end-to-end verificationdocs/                      # per-epic design notes, verification reports, runbooks (see index below)

Set up a dev environment

Prerequisites: Python 3.11+, uv (no uv? python -m venv + pip install -e works as a substitute for the uv commands below).

git clone <this repo's URL> memory-core && cd memory-coreuv venv --python 3.11 .venvsource .venv/bin/activate          # Windows: .venv\Scripts\activate
# Pick dependency groups as needed — they can be combined, no need to install everything:uv pip install -e ".[dev]"                 # required to run tests/lintuv pip install -e ".[llm,embedding]"       # required for the "minimal working memory system" (see the demo below)uv pip install -e ".[mcp]"                 # extra: connecting to Claude Desktop/Cursoruv pip install -e ".[rl]"                  # extra: GRPO training (requires a GPU + CUDA)uv pip install -e ".[cloud]"               # extra: cloud API / Postgres backenduv pip install -e ".[multimodal]"          # extra: image + CLIP verification

The embedding group downloads the default model BAAI/bge-m3 from HuggingFace on first use (~2.2GB) — make sure huggingface.co is reachable (if you're behind the Great Firewall, export HF_ENDPOINT=https://hf-mirror.com routes through a mirror). You can also swap in a smaller model via export EMBEDDING_MODEL=<any sentence-transformers model name>.

Run a minimal example (no MCP, plain Python calls)

With dev + llm + embedding installed and the three env vars from "Get started" set, this script exercises the full "write a memory → retrieve it" pipeline directly (the same logic behind mcp_server/server.py's add_memory/search_memory tools, just calling the library directly instead of going through the MCP protocol layer):

# demo.pyfrom memory_core.graph.local_store import LocalGraphStorefrom memory_core.graph.incremental import IncrementalIngestorfrom memory_core.llm.openai_compatible import OpenAICompatibleProviderfrom memory_core.llm.local_sentence_transformer import SentenceTransformerProviderfrom memory_core.memory_manager.policy import RuleBasedPolicyfrom memory_core.retrieval.ppr import personalized_pagerank, rank_entitiesfrom memory_core.retrieval.query_match import match_query_to_triplesfrom memory_core.retrieval.ranker import build_context
store = LocalGraphStore("demo.sqlite3")          # a local file; delete it to resetllm = OpenAICompatibleProvider()                 # reads LLM_API_KEY etc. from the environmentembedder = SentenceTransformerProvider()         # downloads bge-m3 on first run
# 1. Write a memory: the LLM extracts triples, incrementally merged into the graphingestor = IncrementalIngestor(store, llm, policy=RuleBasedPolicy())result = ingestor.ingest("I do AI research at CAS, mostly in Python.", source_id="demo")print(f"added {result.new_entities} entities, {result.new_relations} relations")
# 2. Retrieve: match the query against triples -> PPR diffusion -> assemble a natural-language contextquery = "Where do I work?"entities, relations = store.all_entities(), store.all_relations()entities_by_id = {e.id: e for e in entities}matches = match_query_to_triples(query, relations, entities_by_id, embedder, top_k=10)seed_ids = {r.relation.subject_id for r in matches} | {r.relation.object_id for r in matches}scores = personalized_pagerank(entities, relations, seed_entity_ids=list(seed_ids))ranked_ids = [eid for eid, _ in rank_entities(scores)]print(build_context(relations, entities_by_id, ranked_ids, top_k=10))

python demo.py

Here's real output from a live run against DeepSeek with the exact input shown above (not fabricated, not cleaned up — this is what actually came back):

added 3 entities, 2 relationsI do AI research at CAS (recorded at 2026-09-05 10:40:00).I do AI research mostly in Python (recorded at 2026-09-05 10:40:00).

Exact wording and entity/relation counts depend on the LLM's own extraction and will vary between runs, but as long as the env vars are set correctly, non-empty output means the pipeline works end to end. retrieval/ranker.py detects whether a relation's text is CJK or not and renders it accordingly (no spaces + a Chinese timestamp label for CJK, spaced words + an English timestamp label otherwise), so English input no longer comes out as one run-on word like earlier versions of this demo did.

Testing

pytest                    # everythingpytest -m "not slow"      # skip tests that download models / train — runs in seconds

Most of the "slow" tests aren't mocked — they're real calls (real LLM API, real local embedding model, real CLIP model) and need the corresponding env vars (LLM_API_KEY, etc.) or an already-downloaded model cache.

Documentation index

Every doc above now has both a Chinese and an English version.

Known limitations

• Multimodal verification for voice input (ASR + audio embedding) isn't implemented yet.
• The GRPO training dataset (140 real samples) and the number of training steps are still small; memory_manager_eval.md honestly documents how that limits training effectiveness.
• The cloud API/billing is skeleton-level only and hasn't been connected to a real production environment.

License

See LICENSE.