io.github.TeleAI-UAGIPublished by GitHub account TeleAI-UAGIv1.10.1Updated Oct 30, 2026
Long-term and multimodal memory for AI agents - character-aware, mem0-compatible, fully-local option
TeleMem gives an assistant a long-term memory store: save facts or conversations, search them semantically, and manage or delete stored memories.
Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.
Follow the launch instructions in the repository.
[arXiv] [CI] [PyPI] [GitHub Stars] [License: Apache 2.0] [Last Commit] [PRs Welcome] [MCPVault: verified]
If you find this project helpful, please give us a ⭐️ on GitHub for the latest update.
🤝 Contributions welcome! Feel free to open an issue or submit a pull request.
TeleMem is an agent memory management layer that can be used as a high-performance drop-in replacement for Mem0 with one line of code (import telemem as mem0), deeply optimized for complex scenarios involving multi-turn dialogues, character modeling, long-term information storage, and semantic retrieval.
Through its unique context-aware enhancement mechanism, TeleMem provides conversational AI with core infrastructure offering higher accuracy, faster performance, and stronger character memory capabilities.
Building upon this foundation, TeleMem implements video understanding, multimodal reasoning, and visual question answering capabilities. Through a complete pipeline of video frame extraction, caption generation, and vector database construction, AI Agents can effortlessly store, retrieve, and reason over video content just like handling text memories.
The ultimate goal of the TeleMem project is to use an agent's hindsight to improve its foresight.
TeleMem, where memory lives on and intelligence grows strong.
add() / search() accept the same arguments and return the same {"results": [...]} shapes, so existing Mem0 code keeps working.uvx, registers all 8 memory tools under mcp__telemem__*, and securely forwards provider configuration. See the MCP server docs.infer=False/prompt/memory_type now fully honored, offline contract test suite, telemetry disabled by default, and a multi-NPC demo!uvx telemem! Also new: evaluation principles and a LongMemEval harness with built-in baselines.pip install telemem! v1.6.0 adds Ollama/DeepSeek/Kimi configs, LangChain & LlamaIndex examples, and a documentation site.TeleMem enables conversational AI to maintain stable, natural, and continuous worldviews and character settings during long-term interactions through a deeply optimized pipeline of character-aware summarization → semantic clustering deduplication → efficient storage → precise retrieval.
Multi-character virtual agent systems
Long-memory AI assistants (e.g., customer service, companionship, creative co-pilots)
Complex narrative/world-building in virtual environments
Dialogue scenarios with strong contextual dependencies
Video content QA and reasoning
Multimodal agent memory management
Long video understanding and information retrieval
TeleMem deeply refactors Mem0 to address characterization, long-term memory, and high performance. Key differences:
We evaluate the ZH-4O Chinese long-character dialogue dataset constructed in the paper MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialogues:
Memory capability was assessed via QA benchmarks, e.g.:
LLM: Qwen3-8B (thinking mode disabled)
Embedding model: Qwen3-Embedding-8B
Metric: QA accuracy
Using uv (recommended — creates .venv from the committed uv.lock for a reproducible environment):
Or with conda + pip:
Set your OpenAI API key:
Memory() uses the default provider settings inherited from mem0ai. To use the repository's local Qwen + FAISS configuration, load config/config.yaml explicitly:
The runnable examples also honor the same configuration through TELEMEM_CONFIG:
TeleMem supports MiniMax as an LLM backend via its OpenAI-compatible API.
A ready-to-use example config is provided at config/config.minimax.yaml.
Key points for MiniMax usage:
MiniMax-M3.1-Flash (default) or MiniMax-M3, both with a 1,000,000-token context window and text/image/video input. Existing configs keep their explicit model IDs, including legacy and custom models.https://api.minimax.io/v1 (global) or https://api.minimax.cn/v1 (China) as openai_base_url; existing custom endpoint values are preserved.max reasoning effort when omitted. The preset sets is_reasoning_model: true so mem0 forwards an optional reasoning_effort (low, medium, high, xhigh, or max) and omits its small default token cap. Flash rejects disabled thinking, reasoning_effort="none", service_tier, and reasoning_split=false. See the OpenAI-compatible API contract for details.response_callback to capture the complete SDK assistant message and append it with tool results on the next call. Preserve both reasoning_content and tool_calls; the adapter's simplified return value is not a complete assistant message.text-embedding-3-small) in the embedder sectionTeleMem works with any OpenAI-compatible endpoint. Ready-to-use config examples ship in config/:
The add() method injects one or more dialogue turns into the memory system.
Returns the mem0-compatible shape: {"results": [{"id": "...", "memory": "...", "event": "ADD"}, ...]}
add()🎭 Multi-character demo: examples/multi_npc.py runs five tavern NPCs through one scene — a single
add_batch(scene, user_id=[...])call gives each NPC a private memory profile plus a shared"events"world-state, and each NPC then recalls the scene from their own perspective.
Performs semantic vector-based retrieval of relevant memories with context-aware recall.
Returns the mem0-compatible shape: {"results": [{"id": "...", "memory": "...", "score": ..., ...}, ...]}
🔍 Search is based on FAISS vector retrieval, supporting millisecond-level responses.
Beyond text memory, TeleMem further extends multimodal capabilities. Drawing inspiration from Deep Video Discovery's Agentic Search and Tool Use approach, we implemented two core methods in the TeleMemory class to support intelligent storage and semantic retrieval of video content.
decode_video_to_frames - Decodes video to JPEG frames at configured FPSprocess_video - Uses VLM (e.g., Qwen3-Omni) to generate detailed descriptions for each clipinit_single_video_db - Generates embeddings for semantic retrieval💡 Smart Caching: If the target file for a stage already exists, that stage is automatically skipped to save computational resources.
search_mm internally uses MMCoreAgent, employing a THINK → ACTION → OBSERVATION loop with three specialized tools:
Run the multimodal demo:
On the first run, frames, captions and VDB JSON will be generated under the chosen output_dir. The repository ships a small sample video; generating captions and the video database still requires configured VLM and embedding services unless you already have these artifacts locally.
Complete code example:
TeleMem ships a Model Context Protocol (MCP) server, so any MCP-compatible client — Claude Desktop, Claude Code, Cursor, custom agents — can use TeleMem as its long-term memory.
Built on the official MCP Python SDK v2, the server implements the current MCP specification (2026-07-28) while remaining compatible with older clients; every tool declares titles, behavior annotations (read-only/destructive hints), and structured output.
The server exposes eight tools: add_memory, search_memories, get_memories, get_memory, update_memory, delete_memory, delete_all_memories, and memory_history. Calls without an explicit scope default to TELEMEM_DEFAULT_USER_ID (telemem-mcp); destructive bulk deletion always requires an explicit scope.
Claude Desktop / Cursor configuration (examples/mcp_config.json):
Or drive it programmatically over stdio — the quickstart flow as MCP tool calls:
See docs/MCP.md for the full tool reference, transports, and client setup.
TeleMem drops into any agent framework with the same two calls — search() before answering, add() after each exchange:
Because TeleMem is mem0 API-compatible, any framework adapter written for Mem0's OSS client also works — point it at telemem.Memory instead.
TeleMem automatically creates a structured storage layout under ./faiss_db/, organized by session and character:
All memories include summary, round number, timestamp, and character, facilitating auditing and debugging.
TeleMem generates video-related storage files in the .data/samples/video/ directory:
TeleMem itself collects no telemetry. The underlying mem0ai library ships
anonymized PostHog usage telemetry, which TeleMem disables by default
(import telemem sets MEM0_TELEMETRY=False unless you have already set it).
To opt back in:
uv run pytest tests/ -q) on Python 3.10–3.12 for every PR.TeleMem’s development has been deeply inspired by open-source communities and cutting-edge research. We extend our sincere gratitude to the following projects and teams:
If you find TeleMem useful to your research or development, please cite our arXiv paper:
Chunliang Chen, Ming Guan, Xiao Lin, Jiaxu Li, Luxi Lin, Qiyi Wang, Xiangyu Chen, Jixiang Luo, Changzhi Sun, Dell Zhang, Xuelong Li. TeleMem: Building Long-Term and Multimodal Memory for Agentic AI. arXiv:2601.06037, 2026. https://arxiv.org/abs/2601.06037
Citation metadata is also available in CITATION.cff (GitHub's "Cite this repository" button).
If you find this project helpful, please give it a ⭐️ — starring also keeps updates in your GitHub feed.
Made with ❤️ by Bloo-Mind AI Ltd and the Ubiquitous AGI team at TeleAI.
mcp-name: io.github.TeleAI-UAGI/telemem
Source: README.md at commit 97fd2b9
Add memory
Store a fact, preference, or conversation in TeleMem long-term memory. Provide `text` for a single statement or `messages` for conversation turns. Scoped by user_id/agent_id/run_id; defaults to the server's default user.
textinferrun_iduser_idagent_idmessagesmetadataSearch memories
Semantic search over TeleMem memories. Returns the matching memories as one consolidated text passage (TeleMem fuses related memories). Shared event memories (pseudo-user "events") are searched automatically.
limitqueryrequiredrun_iduser_idagent_idthresholdList memories
List stored memories for a user/agent/run with their memory_ids. Use user_id "events" to list shared conversation-event memories.
limitrun_iduser_idagent_idGet memory
Fetch a single memory by its memory_id.
memory_idrequiredUpdate memory
Overwrite the text of an existing memory by memory_id.
textrequiredmetadatamemory_idrequiredDelete memory
Delete a single memory by memory_id.
memory_idrequiredDelete all memories in scope
Delete every memory in the given scope. Destructive: requires an explicit user_id, agent_id, or run_id — the default user is never assumed.
run_iduser_idagent_idMemory history
Show the change history (ADD/UPDATE/DELETE events) of a memory.
memory_idrequired90% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.
flowchart LR A["Dialogue<br/>messages"] --> B["Character-aware<br/>summarization<br/>(global + per-character)"] B --> C["Embedding +<br/>similar-memory<br/>retrieval"] C --> D["Write buffer<br/>(batch flush)"] D --> E["LLM semantic<br/>clustering & fusion"] E --> F[("FAISS index +<br/>JSON metadata")] Q["Query"] --> S["Vector search<br/>+ rerank"] F --> S S --> R["results"]{"question": "What is Zhao Qi's nickname for Bai Yulan? A Xiaobai B Xiaoyu C Lanlan D Yuyu","answer": "A"},{"question": "What is the relationship between Zhao Qi and Bai Yulan? A Classmates B Teacher and student C Enemies D Neighbors","answer": "B"}pip install telemem # core (text memory)pip install "telemem[mcp]" # + MCP serverpip install "telemem[video]" # + video/multimodal pipelinepip install "telemem[all]" # everythinguv sync --all-extras # install TeleMem (editable) + all extras, incl. MCPuv run python examples/quickstart.py# Create and activate virtual environmentconda create -n telemem python=3.10conda activate telemem# Install from source (editable), with the extras you needpip install -e ".[all]"export OPENAI_API_KEY="your-openai-api-key"# python examples/quickstart.pyimport telemem as mem0
memory = mem0.Memory()
messages = [ {"role": "user", "content": "Jordan, did you take the subway to work again today?"}, {"role": "assistant", "content": "Yes, James. The subway is much faster than driving. I leave at 7 o'clock and it's just not crowded."}, {"role": "user", "content": "Jordan, I want to try taking the subway too. Can you tell me which station is closest?"}, {"role": "assistant", "content": "Of course, James. You take Line 2 to Civic Center Station, exit from Exit A, and walk 5 minutes to the company."}]
memory.add(messages=messages, user_id="Jordan")results = memory.search("What transportation did Jordan use to go to work today?", user_id="Jordan")for hit in results["results"]: # same result shape as mem0 print(hit["memory"])from telemem.utils import load_configimport telemem as mem0
config = load_config("config/config.yaml")memory = mem0.Memory(config=config)TELEMEM_CONFIG=config/config.yaml python examples/quickstart.pyexport MINIMAX_API_KEY="your-minimax-api-key"export OPENAI_API_KEY="your-openai-api-key" # still needed for embeddingsfrom telemem.utils import load_configimport telemem as mem0
config = load_config("config/config.minimax.yaml")memory = mem0.Memory(config=config)TELEMEM_CONFIG=config/config.ollama.yaml python examples/quickstart.py # 100% local memorytelemem/├── assets/ # Documentation assets and figures├── baselines/ # Baseline implementations for comparative evaluation│ ├── RAG # Retrieval-Augmented Generation baseline│ ├── MemoBase # MemoBase memory management system│ ├── MOOM # MOOM dual-branch narrative memory framework│ ├── A-mem # A-mem agent memory baseline│ └── Mem0 # Mem0 baseline implementation├── config/ │ ├── config.yaml # TeleMem default configuration│ └── config.minimax.yaml # MiniMax provider example configuration├── data/ # Small sample datasets for evaluation or demonstration├── examples/ # Code examples and tutorial demos│ ├── quickstart.py # Quick start│ ├── quickstart_mm.py # Quick start (multimodal)│ ├── mcp_client.py # Quick start over MCP (stdio client)│ ├── mcp_config.json # MCP config snippet for Claude Desktop / Cursor│ └── deepseek-harness.cordis.yml # DeepSeek Harness memory patch├── docs/│ ├── MCP.md # MCP server reference│ └── TeleMem_Tech_Report.pdf├── telemem/ # Telemem code│ └── mcp/ # Model Context Protocol server├── tests/ # Telemem test├── README.md # English README├── README-ZH.md # Chinese README└── pyproject.toml # Python environmentdef add( self, messages, *, user_id: Optional[str] = None, agent_id: Optional[str] = None, run_id: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, infer: bool = True, memory_type: Optional[str] = None, prompt: Optional[str] = None, batch: bool = False,)def search( self, query: str, *, user_id: Optional[str] = None, agent_id: Optional[str] = None, run_id: Optional[str] = None, limit: int = 100, filters: Optional[Dict[str, Any]] = None, threshold: Optional[float] = None, rerank: bool = True,)def add_mm( self, video_path: str, output_dir: str, clip_secs: int | None = None, emb_dim: int | None = None, subtitle_path: str | None = None,){ "output_dir": "/abs/path/to/output_dir"}def search_mm( self, question: str, output_dir: str, max_iterations: int = 15,)python examples/quickstart_mm.pyimport telemem as mem0from pathlib import Pathfrom telemem.mm_utils.core import extract_choice_from_msg
# Initializememory = mem0.Memory()
# Define pathsrepo_root = Path(__file__).resolve().parents[1]video_path = repo_root / "data" / "samples" / "video" / "3EQLFHRHpag.mp4"video_name = video_path.stemoutput_dir = video_path.parent
# Step 1: Add video to memory (auto-processing)vdb_json_path = output_dir / "vdb" / video_name / f"{video_name}_vdb.json"if not vdb_json_path.exists(): result = memory.add_mm( video_path=str(video_path), output_dir=str(output_dir), ) print(f"Video processing complete: {result}")else: print(f"VDB already exists: {vdb_json_path}")
# Step 2: Query video contentquestion = """The problems people encounter in the video are caused by what?(A) Catastrophic weather.(B) Global warming.(C) Financial crisis.(D) Oil crisis."""
messages = memory.search_mm( question=question, output_dir=str(output_dir), max_iterations=15,)
# Extract final answeranswer = extract_choice_from_msg(messages)print(f"Answer: ({answer})")pip install telemem
telemem-mcp # stdio (default)telemem-mcp --transport streamable-http # Streamable HTTP on :8421TELEMEM_CONFIG=config/config.yaml telemem-mcp # custom TeleMem configuvx telemem # zero-install run (stdio){ "mcpServers": { "telemem": { "command": "telemem-mcp", "env": { "TELEMEM_CONFIG": "/absolute/path/to/config/config.yaml", "OPENAI_API_KEY": "sk-..." } } }}python examples/mcp_client.pyfaiss_db/├── session_001_events.index├── session_001_events_meta.json ├── session_001_person_1.index ├── session_001_person_1_meta.json ├── session_001_person_2.index └── session_001_person_2_meta.json { "summary": "Characters discussed the upcoming action plan.", "sample_id": "session_001", "round_index": 3, "timestamp": "2024-01-01T00:00:00Z", "user": "Jordan" // Only present in person_*.json}video/├── frames/│ └── <video_name>/│ └── frames/│ ├── frame_000001_n0.00.jpg│ ├── frame_000002_n0.50.jpg│ └── ...├── captions/│ └── <video_name>/│ ├── captions.json # Clip descriptions + subject registry│ └── ckpt/ # Checkpoint for resume│ ├── 0_10.json│ └── 10_20.json└── vdb/ └── <video_name>/ └── <video_name>_vdb.json # Semantic retrieval vector database{ "0_10": { "caption": "The narrator discusses climate data, showing melting glaciers..." }, "10_20": { "caption": "Scene shifts to coastal communities affected by rising sea levels..." }, "subject_registry": { "narrator": { "name": "narrator", "appearance": ["professional attire"], "identity": ["climate scientist"], "first_seen": "00:00:00" } }}export MEM0_TELEMETRY=true@article{chen2026telemem, title = {TeleMem: Building Long-Term and Multimodal Memory for Agentic AI}, author = {Chen, Chunliang and Guan, Ming and Lin, Xiao and Li, Jiaxu and Lin, Luxi and Wang, Qiyi and Chen, Xiangyu and Luo, Jixiang and Sun, Changzhi and Zhang, Dell and Li, Xuelong}, journal = {arXiv preprint arXiv:2601.06037}, year = {2026}, url = {https://arxiv.org/abs/2601.06037}}