Context Engine

io.github.melonelishPublished by GitHub account melonelishv0.1.0Updated Oct 24, 2026

Compress logs, retrieval chunks, and code context into structured LLM-ready signal.

VerifiedSTDIODesktop onlyKnowledge & MemoryDeveloper ToolsAI & ML

Overview

AI-generated overview

Compresses logs, RAG retrieval chunks, and code context into structured, LLM-ready input through a single compress_context tool.

What you can do
Context Engine is a task-aware context compression layer exposed over stdio as one MCP tool, compress_context, which takes a mode, a budget, and either raw content or a structured payload. Three modes are supported: logs, which keeps root-cause lines and traceback tails while folding repeated heartbeat, retry, and duplicate noise into counts; rag, which reorders retrieved chunks around a question into high-signal, supporting, and low-signal tiers; and code, which ranks hotspot files using issue terms, test failure output, and path hints. Results come back in a structured envelope with a summary, key facts, discarded noise, and an llm_ready_context block, or a structured error with an error…
When to use it
Use it when an agent, RAG pipeline, or coding assistant is being handed context that is too long, too noisy, or poorly ordered, such as long logs with repeated noise, retrieval results dumped in recall order, or a pile of files with no clear fix target. It is aimed at early external trials, integration testing, and developer workflow validation rather than production deployments.
Requirements
Runs locally as a Python package (melonelish-context-engine) over stdio, started with the context-engine-mcp command. Requires Python 3.11 to 3.13 and pip, with the optional mcp extra installed for the MCP server. No authentication, environment variables, or headers are declared. Input limits apply: 200000 characters per text segment, 64 structured list items, and 2000000 bytes per input file.
Before you install
The README states there is no production-grade authentication, persistence, monitoring, or audit capability, and no long-term compatibility commitment across future major versions. It is described as an early beta at v0.1.0 with a small benchmark set. Binary inputs such as PDFs, images, and Office files are not supported. The server reads the context content you submit, so avoid passing secrets or sensitive data unless you accept that it is processed locally by this tool.

Installation

In SourceWeft

  1. Open Context Engine in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

Context Engine - 面向 Agent / RAG / AI Coding 的上下文压缩引擎

把噪声日志、检索片段和代码上下文，压缩成大模型更容易使用的高信号输入。

面向 Agent 工作流、RAG 管线、AI Coding 助手的任务感知 Context Compression Layer。

[Python] [Typer CLI] [MCP] [License]

PyPI · GitHub · MCP Registry

为什么需要 Context Engine

很多 Agent / RAG / AI Coding 系统的问题，不是“没有上下文”，而是“上下文太脏、太长、太乱”。

日志里有大量重复 heartbeat 和 retry；RAG 检索结果里混着相关与不相关 chunk；代码修复场景里，模型经常拿到一堆文件，却缺少最小修复线索。

Context Engine 解决的是这个中间层问题：

在把内容交给大模型之前，先按任务类型压缩、排序、去噪，并输出结构化的高信号上下文。

它不是普通摘要工具，而是一个更偏工程化的上下文整理层。

能力边界问题

场景直接丢给模型Context Engine
长日志 / traceback复制全部日志，或者简单截断保留疑似根因、traceback 尾部，折叠重复噪声
RAG 检索结果按检索顺序塞入所有 chunk围绕用户问题重新排序，区分高信号与低信号证据
AI Coding 修复把附近文件全部塞进去根据 issue / test output 排序 hotspot file
输入异常抛原始 traceback返回结构化错误、错误码和修复提示
Token 不够从头或从尾硬切在标准化和去重后按预算裁剪

系统架构

核心理念：先把上下文变成可解释、可预算、可消费的结构，再交给 LLM。

mermaid
flowchart TB    classDef input fill:#fff7d6,stroke:#d99b00,stroke-width:2px,color:#5c3b00;    classDef core fill:#e7f1ff,stroke:#2563eb,stroke-width:2px,color:#123c8c;    classDef mode fill:#eaf8ef,stroke:#1f8a4c,stroke-width:2px,color:#0f4f2b;    classDef output fill:#f5ecff,stroke:#7c3aed,stroke-width:2px,color:#3f1d7a;
    subgraph Inputs["输入层"]        CLI["CLI<br/>本地文件 / 示例"]:::input        SDK["Python API<br/>CompressionRequest"]:::input        MCP["MCP Tool<br/>compress_context"]:::input    end
    subgraph Core["核心 Pipeline"]        Validate["输入校验<br/>schema + size guard"]:::core        Normalize["标准化<br/>ContextItem list"]:::core        Dedupe["去重<br/>重复内容折叠"]:::core        Budget["预算控制<br/>small / medium / large"]:::core    end
    subgraph Compressors["任务感知压缩器"]        Logs["logs<br/>root cause + traceback"]:::mode        Rag["rag<br/>question-aware evidence"]:::mode        Code["code<br/>hotspot files + failure signal"]:::mode    end
    subgraph Outputs["输出契约"]        Result["ok: true<br/>summary + key facts + llm_ready_context"]:::output        Error["ok: false<br/>error_code + hint + details"]:::output    end
    CLI --> Validate    SDK --> Validate    MCP --> Validate    Validate --> Normalize --> Dedupe --> Budget    Budget --> Logs    Budget --> Rag    Budget --> Code    Logs --> Result    Rag --> Result    Code --> Result    Validate --> Error

数据流

mermaid
sequenceDiagram    autonumber    participant U as 用户 / Agent    participant C as CLI / MCP / SDK    participant V as Validator    participant P as Pipeline    participant M as Mode Compressor    participant O as Output Envelope
    U->>C: 提交 logs / rag / code 输入    C->>V: 校验字段、大小和模式    V->>P: 标准化为 ContextItem    P->>P: 去重、排序、预算裁剪    P->>M: 进入任务压缩器    M->>O: 生成 summary / key_facts / llm_ready_context    O-->>U: 返回结构化结果或结构化错误

核心能力

1. logs：根因导向的日志压缩

logs 模式面向长日志、重复日志、异常栈和混合噪声。

能力说明
根因提取优先保留 error、exception、failed、fatal 等关键行
Traceback 保留保留更接近根因的 traceback 尾部
噪声折叠把重复 heartbeat、poll、retry、duplicate line 归并成计数
LLM-ready 输出生成可以直接放进诊断或修复 prompt 的上下文块

2. rag：围绕问题的证据重排

rag 模式接收 question 和 chunks，把检索结果从“按召回顺序堆叠”变成“围绕问题排序”。

分层含义
HIGH与问题重叠度高，可能直接支持回答
SUPPORT有帮助但不是核心证据
Low signal相关性弱，不应该占据主要上下文窗口

3. code：最小修复上下文

code 模式接收 issue、可选 test_output 和 files，用于给 AI Coding 助手准备更聚焦的修复输入。

信号作用
Issue terms保留用户描述的问题焦点
Failure terms把测试失败、异常信息和候选文件关联起来
Hot path hints对 test、parser、pipeline、service、validator 等路径加权
Supporting files保留辅助上下文，但不让它淹没主线

4. CLI + MCP 双入口

同一套压缩逻辑可通过三种方式使用：

入口用途
Python API集成到自己的包或服务里
CLI本地处理日志、样例和脚本任务
MCP Server接入支持 MCP 的 Agent / IDE / 自动化环境

快速开始

前置要求

项目要求
Python3.11 到 3.13
包管理器pip
MCP 运行时可选，通过 mcp extra 安装

安装

powershell
python -m venv .venv.venv\Scripts\Activate.ps1python -m pip install -e .[dev,mcp]

从 PyPI 安装发布包：

powershell
python -m pip install "melonelish-context-engine[mcp]"

运行示例

powershell
python -m pytest -qcontext-engine --mode logs --input examples/logs/sample.log --budget mediumcontext-engine --mode rag --input examples/rag/sample.json --budget mediumcontext-engine --mode code --input examples/code/sample.json --budget medium

CLI 使用

powershell
context-engine --mode logs --input examples/logs/sample.log --budget smallcontext-engine --mode rag --input examples/rag/sample.json --budget mediumcontext-engine --mode code --input examples/code/sample.json --budget large

输入格式

logs：纯文本
text
2026-07-03 10:01:12 ERROR payment.worker failed to charge orderTraceback (most recent call last):  ...ValueError: missing customer_id
rag：JSON
json
{  "question": "Why did checkout fail?",  "chunks": [    {      "content": "Checkout fails when customer_id is missing.",      "metadata": {"source": "runbook.md"}    }  ]}
code：JSON
json
{  "issue": "Checkout test fails when customer_id is omitted.",  "test_output": "ValueError: missing customer_id",  "files": [    {      "path": "src/payments/checkout.py",      "content": "def checkout(order): ..."    }  ]}

MCP 使用

通过 stdio 启动 MCP Server：

powershell
context-engine-mcp

当前暴露一个工具：

工具参数功能
compress_contextmode、budget、content 或 payload压缩 logs / rag / code 上下文

示例错误返回：

json
{  "ok": false,  "error": {    "error_code": "invalid_field",    "message": "Field 'chunks' must be a non-empty list.",    "hint": "Provide at least one item in 'chunks'."  }}

输出契约

CLI 和 MCP 都返回统一结构：

结果结构
成功{ "ok": true, "result": ... }
失败{ "ok": false, "error": { "error_code": "...", "message": "...", "hint": "...", "details": ... } }

成功结果会包含 schema version、summary、key facts、被丢弃或降权的噪声，以及 llm_ready_context。

安全边界

当前内置限制：

限制项当前值
单段文本最大长度200000 字符
结构化列表最大数量64 项
输入文件最大大小2000000 bytes
Schema version1.0

这些限制用于避免外部工作流把任意超大 payload 直接打进工具。

当前已支持

  • Python 3.11 到 3.13
  • logs / rag / code 三种模式
  • 三种模式的 CLI 使用
  • 通过 compress_context 暴露 MCP 工具
  • 纯文本日志输入
  • JSON 格式的 RAG 和 code 输入
  • 结构化成功 / 失败 envelope
  • 基础 benchmark 和 GitHub Actions CI

暂不支持

  • PDF、图片、Office 文件等二进制输入
  • embedding-aware reranking
  • 仓库级依赖图分析
  • 跨未来大版本的长期兼容性承诺
  • 生产级鉴权、持久化、监控和审计能力

Benchmark

Benchmark 资料位于：

当前样例集体现的行为：

模式对比对象当前效果
logs原始日志 / 简单截断保留根因，折叠重复噪声
rag直接 dump 检索 chunk围绕问题排序证据
code普通文件摘要保留 issue、失败信号、hotspot file 和支持上下文

v0.1.0 的 benchmark 还很小，适合作为回归检查和展示样例，不代表完整生产评测。

开发

powershell
python -m pip install -e .[dev,mcp]python -m pytest -qpython benchmarks/generate_benchmarks.py

CI 会在 push 和 pull request 时运行安装、测试和 benchmark 生成检查。

发布状态

当前版本目标：v0.1.0

已发布渠道：

这个版本适合早期外部试用、集成测试和开发者工作流验证。它已经具备可复用包结构、测试、文档，以及可直接安装的 PyPI / MCP Registry 发布入口，但仍然是范围明确的早期 beta。

Roadmap

  • 引入 embedding 或 reranker 驱动的 RAG 排序。
  • 加强 code hotspot 识别，加入更可靠的结构化代码信号。
  • 扩展 benchmark，加入更多真实脏数据样本。
  • 发布更易安装的正式包版本。
  • 增加更多 Agent Runtime / MCP 集成示例。

License

本项目采用 MIT License。

Source: README.md at commit 288ad17

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.1.0LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts