dompruner-mcp
한국어 | English
[DOM Tree Pruning for DomPruner]
DOM AST middleware for LLM web pipelines — strips layout noise (nav, scripts, sidebars) and passes original text directly. Add a query to filter to relevant sections with BM25.
When an LLM uses a web fetch tool, the raw page HTML is converted to text and passed into the model's context — often tens of thousands of tokens of navigation, sidebars, and boilerplate the model doesn't need. DomPruner strips that at the DOM level and passes only the original content directly to the model.
93.5% fewer context tokens than WebFetch on average. 45% faster end-to-end.
→ Full benchmark
Quick Start
No installation, no API key:
Claude Code
Add to .mcp.json in your project root, or ~/.claude/.mcp.json for global. Run /mcp to verify.
Claude Desktop
Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):
Cursor / Windsurf / other MCP clients
Remote HTTP (no install, always up to date)
For clients that support HTTP transport — no Node.js install required, always runs the latest version:
LangChain / LangGraph
langchain-mcp-adapters wraps any MCP stdio server as LangChain tools automatically:
Ensuring Your AI Always Uses DomPruner
DomPruner's tool description already tells clients to prefer dompruner_fetch over WebFetch. If your client still falls back, add this to its instruction file:
Tools
→ Full tool reference
Benchmark Summary
vs WebFetch (live comparison, Claude Haiku)
Measured using Anthropic's web_fetch_20260209 server tool under identical conditions (same model, system prompt, tool definitions). → Full methodology
BM25 Section Accuracy (snapshot-based, deterministic)
headingBoost=2.0 was selected by running a parameter sweep (1.0–3.0) on the tuning set. The validation set uses five sites not present in the tuning set (react.dev, typescriptlang.org, expressjs.com, doc.rust-lang.org, nextjs.org) to confirm the result is not overfit. The 2 %p gap is within the known heading-bias limitation.
→ Full benchmark · Architecture
vs Crawl4AI
Crawl4AI is a crawler framework — browser automation, JS rendering, click/scroll interaction, JSON extraction, deep crawl. DomPruner is LLM context middleware — a single-purpose layer that reduces token cost when an agent already knows the URL.
They solve different problems. But both can be used to read a web page inside an MCP agent, so the comparison is real.
Setup
Held-out validation: 5 sites not in DomPruner's tuning set
The most defensible comparison: five sites that were not used to tune DomPruner's parameters (react.dev, typescriptlang.org, expressjs.com, doc.rust-lang.org, nextjs.org). Both tools use the same query.
On sites DomPruner was not tuned on: 1.9× fewer tokens with the same accuracy (96%).
Additional reference: 10-page tuning set (MDN, docs.python.org)
On DomPruner's own tuning pages, vs Crawl4AI PruningContentFilter: 13.2× smaller, 8 pp more accurate (90% → 98%). Crawl4AI BM25ContentFilter with the same query returns near-empty output on 5 of 10 pages — a structural issue with <dl>/<dt>/<dd> definition-list pages where a threshold sweep (0.0–0.10) shows a hard cliff at 0.01 with no intermediate optimum.
→ Full comparison with all tables and threshold sweep data
Related
- dompruner-py — Python port.
DomPrunerLoader, DomPrunerSitemapLoader, DomPrunerFetchTool for LangChain. pip install dompruner.
- LangChain integrations — dompruner-py listed as a third-party web loader.
Glama Score
[dompruner-mcp MCP server]
License
MIT