LatentGate

See it work

Try it in your browser → The real optimizer runs on your machine via WebAssembly; nothing you paste is uploaded.

A 400-line error log, read through LatentGate (balanced, measured with tiktoken):

BEFORE  13,641 tokens2026-10-08 09:00:00 ERROR order 10000 failed: payment gateway timeout after 2982ms (retry 1/3)... 398 more lines in the same shape ...
AFTER   145 tokens  (98.9% saved)2026-10-08 09:00:00 INFO app started, 3 workers2026-10-08 09:00:00 ERROR order 10000 failed: payment gateway timeout after 2982ms (retry 1/3)[… 398 similar lines: order 10001…10398, 2900…3100 ms …]2026-10-08 09:06:39 ERROR order 10399 failed: payment gateway timeout after 3030ms (retry 1/3)2026-10-08 09:07:00 FATAL circuit breaker open for payments-api

Install it into Claude Code in one line, then ask "read app.log and tell me why orders fail":

claude mcp add --scope user latent-gate -- uvx --from "latent-gate[mcp,tokens]" latent-gate-mcp

What you can rely on: the same input always gives the same output (so provider prompt caching keeps working); output is never larger than input; code blocks, URLs and quoted strings come back byte-for-byte; level="lossless" changes nothing but whitespace, JSON formatting and exact duplicates. What you trade at the default level: runs of near-identical lines are folded to first, last and value ranges, so individual IDs inside a folded run are not kept. The full corpus, including where it loses facts, is in Cost Benchmarks and reproducible with latent-gate --optimizer-benchmark.

Works in Claude Code, Claude Desktop, Cursor, Cline, Continue and Zed, free and offline.

Don't want to install anything? The official hosted LatentGate on mcprush also runs where local servers can't, such as claude.ai on the web and mobile. It has a free tier (300 calls a month) and paid plans from $9/month.

The Problem

Every time you send an image or long prompt to GPT-4o / Claude / Gemini, you burn 1,000+ tokens on processing that could happen locally for free.

Traditional:  Image -> Cloud LLM (1,200 tokens) -> AnswerLatentGate:   Image -> Local Ollama (FREE) -> Cloud LLM (200 tokens) -> Answer

Features

Use it in Claude

LatentGate plugs into Claude as an MCP server. Its optimizer tools work offline, with no Ollama and no API key — Claude reads big logs, JSON dumps and docs through it and spends a fraction of the context. A 400-line error log goes from 14,400 to ~100 tokens.

Requires uv (uvx fetches LatentGate from PyPI on first run). The first run downloads dependencies, which can exceed Claude's 30-second MCP startup limit on a slow connection; run this once beforehand (later starts take ~2s):

uvx --from "latent-gate[mcp,tokens]" latent-gate --optimizer-benchmark

Claude Code — plugin (MCP server + a skill that tells Claude when to use it):

/plugin marketplace add KathanModh259/latent-gate/plugin install latent-gate@latent-gate

Claude Code — MCP server only:

claude mcp add latent-gate -- uvx --from "latent-gate[mcp,tokens]" latent-gate-mcp

Any agent — from the Agensi marketplace: LatentGate Context Saver is the official skill for Claude Code, Cursor, Codex CLI and 20+ other agents. It sets LatentGate up if it's missing, works with the local and hosted editions, and tells your agent when to use it.

Claude Desktop — add to claude_desktop_config.json and restart:

{  "mcpServers": {    "latent-gate": {      "command": "uvx",      "args": ["--from", "latent-gate[mcp,tokens]", "latent-gate-mcp"]    }  }}

Then just ask: "Read logs/app.log with latent-gate and tell me why orders fail."

Quick Start

Install

# Core installpip install latent-gate
# With MCP server (for Claude Desktop, Cursor, Cline, etc.)pip install latent-gate[mcp]
# With API server (for web applications)pip install latent-gate[api]
# With video processingpip install latent-gate[video]
# With embedding-based similarity (more accurate selective decoding)pip install latent-gate[embeddings]
# With LangChain integrationpip install latent-gate[langchain]
# With AWS Bedrock supportpip install latent-gate[bedrock]
# Exact token counts via tiktoken (otherwise a calibrated ~6%-error estimate)pip install latent-gate[tokens]
# With all featurespip install latent-gate[all]

Pull Ollama Models

ollama pull llava:7b      # Vision model (required for image queries)ollama pull llama3:8b     # Text model (required for text compression & prediction)

One-Command Quickstart

chmod +x scripts/quickstart.sh./scripts/quickstart.sh

This starts everything: Ollama → pulls models → API server → website. See scripts/quickstart.sh for options like --no-pull, --no-website, --port 9000.

CLI Usage

# Image querylatent-gate photo.jpg "What is in this image?" --provider ollama -v
# Text compressionlatent-gate --text "Your long prompt here..." --provider ollama -v
# Text from filelatent-gate --text-file prompt.txt --provider openai -v
# Image + Text combinedlatent-gate photo.jpg "Analyze" --text "Extra context..." -v
# Full JSON outputlatent-gate photo.jpg "Describe" --json -v
# Compress only, deterministically (no LLM call, ~1ms, reproducible)cat prompt.txt | latent-gate --text-file - --compress-only --deterministic --level balanced
# Reproduce the token-savings benchmark (no Ollama needed)latent-gate --optimizer-benchmark
# Production benchmarklatent-gate --benchmark --benchmark-output reports/benchmark.json
# Start API server (requires: pip install latent-gate[api])latent-gate-api

Production Hardening

For API deployments, restrict direct image-path reads to trusted directories:

set LATENTGATE_ALLOWED_IMAGE_ROOTS=C:\safe-images;D:\datasetslatent-gate-api

Benchmark before releases so speed and savings are measured, not guessed:

latent-gate --benchmark --json

Python API

Image Query

from latent_gate import LatentGatePipeline, PipelineConfig
config = PipelineConfig(    vision_model="llava:7b",    predictor_model="llama3:8b",    remote_provider="openai",    remote_model="gpt-4o-mini",)
with LatentGatePipeline(config) as pipeline:    result = pipeline.query("photo.jpg", "What is in this image?")
    print(result["answer"])    print(f"Tokens sent: ~{result['tokens_estimated']}")    print(f"Timing: {result['timing']}")

Text Compression

# Long prompt compressionresult = pipeline.query_text("Your 500-word prompt here...", mode="auto")
# Conversation history compressionmessages = [    {"role": "user", "content": "Help me with Kubernetes setup"},    {"role": "assistant", "content": "Sure! What's your target configuration?"},    {"role": "user", "content": "3 nodes, t3.large, us-east-1 with autoscaling"},]result = pipeline.query_conversation(messages, "Now give me the setup commands")
# RAG document compressiondocuments = ["doc1 text...", "doc2 text...", "doc3 text..."]result = pipeline.query_documents(documents, "How do I implement JWT refresh?")
# Universal (auto-detect input type)result = pipeline.query_universal(text="Explain this code...", image="screenshot.png")

Batch Processing

# Sequential with selective decoding (skips redundant API calls)results = pipeline.query_batch(image_paths, "Describe each scene")
# Parallel processingresults = pipeline.query_batch(image_paths, "Describe each scene", parallel=True, max_workers=4)
# Text batchresults = pipeline.query_batch_texts(text_list, question="Summarize each")

Streaming

# Stream image queryfor token in pipeline.query_stream("photo.jpg", "Describe this"):    print(token, end="", flush=True)
# Stream text queryfor token in pipeline.query_text_stream("Long prompt...", mode="compress"):    print(token, end="", flush=True)

REST API

Start Server

# Default (0.0.0.0:8000)latent-gate-api
# Custom host/port# Linux/macOS:LATENTGATE_HOST=127.0.0.1 LATENTGATE_PORT=9000 latent-gate-api
# Windows PowerShell:$env:LATENTGATE_HOST="127.0.0.1"; $env:LATENTGATE_PORT="9000"; latent-gate-api
# Windows CMD:set LATENTGATE_HOST=127.0.0.1 && set LATENTGATE_PORT=9000 && latent-gate-api

Endpoints

Example Requests

import requests
# Image queryresponse = requests.post("http://localhost:8000/query/image", json={    "image_path": "photo.jpg",    "question": "What is in this image?"})
# Text queryresponse = requests.post("http://localhost:8000/query/text", json={    "text": "Your long prompt here...",    "question": "Summarize this",    "mode": "auto"  # auto | compress | summarize | condense | code})
# Health checkresponse = requests.get("http://localhost:8000/health")print(response.json())  # {"status": "healthy", "ollama_connected": true, ...}

Async Support

import asynciofrom latent_gate import AsyncLatentGatePipeline, PipelineConfig
async def main():    async with AsyncLatentGatePipeline() as pipeline:        # Single queries        result = await pipeline.query("photo.jpg", "What is this?")        result = await pipeline.query_text("Long prompt...")
        # Concurrent batch processing        results = await pipeline.query_many_images(            ["img1.jpg", "img2.jpg", "img3.jpg"],            "Describe each image",            max_concurrent=3,        )
asyncio.run(main())

Video Processing

from latent_gate import LatentGatePipeline, PipelineConfig, VideoProcessor, VideoConfig
config = PipelineConfig(    vision_model="llava:7b",    remote_provider="ollama",    remote_model="llama3:8b",)
video_config = VideoConfig(    fps=1.0,            # Extract 1 frame per second    max_frames=100,     # Max frames to process    quality=95,         # JPEG quality    resize_width=640,   # Resize frames (saves processing time))
with VideoProcessor(config, video_config) as processor:    result = processor.process_video("video.mp4", "Describe the action")
    print(f"Frames processed: {result['total_frames']}")    print(f"Unique scenes: {result['statistics']['unique_scenes']}")    print(f"Skip rate: {result['statistics']['skip_rate']}")

Configuration

Config File

# latentgate.yamlollama_base_url: http://localhost:11434vision_model: llava:7bpredictor_model: llama3:8bremote_provider: openairemote_model: gpt-4o-miniselective_decoding: truesimilarity_threshold: 0.85use_embeddings: trueenable_caching: truetemperature: 0.1request_timeout: 120track_costs: truecost_db_path: "latentgate_costs.db"

from latent_gate import get_config, LatentGatePipeline
config = get_config("latentgate.yaml")with LatentGatePipeline(config) as pipeline:    result = pipeline.query("photo.jpg", "Describe this")

Environment Variables

Save Config

from latent_gate import PipelineConfig, save_config
config = PipelineConfig(remote_provider="anthropic", remote_model="claude-sonnet-5")save_config(config, "my_config.yaml")

Docker

# Start full stack (API + Ollama)docker compose up -d
# Pull models first (one-time setup)docker compose --profile setup up ollama-init
# Start with monitoring (Prometheus + Grafana)docker compose --profile monitoring up -d
# Build and run manuallydocker build -t latent-gate .docker run -p 8000:8000 latent-gate

The docker-compose setup includes:

• latent-gate API server (port 8000)
• Ollama local LLM server (port 11434)
• ollama-init container that auto-pulls required models (profile: setup)
• Prometheus metrics collector on port 9090 (profile: monitoring)
• Grafana dashboard on port 3000 (profile: monitoring, credentials: admin/latentgate)

Monitoring Stack

Start with monitoring:

docker compose --profile monitoring up -d

Access:

• Grafana: http://localhost:3000 (login: admin / latentgate)
• Prometheus: http://localhost:9090
• Metrics endpoint: http://localhost:8000/metrics

The LatentGate dashboard auto-loads in Grafana with 9 panels covering request rate, latency percentiles (p50/p95/p99), token savings, error rates, pipeline health, and endpoint breakdown.

Customize via environment variables:

• GRAFANA_ADMIN — Grafana admin username (default: admin)
• GRAFANA_PASSWORD — Grafana password (default: latentgate)
• GRAFANA_ANONYMOUS — Enable anonymous access (default: true)
• LATENTGATE_ENABLE_METRICS — Enable Prometheus metrics (default: true)

AI Coding Tool Integration (MCP)

LatentGate works as a Model Context Protocol (MCP) server with every major AI coding tool. Your AI assistant automatically compresses images, long prompts, and documents before they reach the cloud model.

Supported Tools

VS Code Extension

code --install-extension KathanModh259.latent-gate-vscode

Features:

• Right-click any image to compress with LatentGate
• Select text and press Ctrl+Shift+Alt+C to compress
• Cost dashboard in activity bar
• Auto-configures MCP for Copilot Chat
• Status bar showing token savings

MCP Setup

For Claude, see Use it in Claude. For Cursor, Cline, Continue, Zed and other MCP clients, use the same server command:

{  "mcpServers": {    "latent-gate": {      "command": "uvx",      "args": ["--from", "latent-gate[mcp,tokens]", "latent-gate-mcp"]    }  }}

Or install it into your environment (pip install "latent-gate[mcp,tokens]") and use "command": "latent-gate-mcp". Note the command is latent-gate-mcp — plain latent-gate is the CLI and will not speak MCP. The image and compress_* tools additionally need ollama pull llava:7b and ollama pull phi3:mini.

See integrations/ folder for detailed setup guides per tool.

Hosted MCP (remote, for gateways like mcprush)

docker build -f Dockerfile.mcp -t latent-gate-mcp .docker run -p 8000:8000 -e LATENTGATE_API_KEY=change-me latent-gate-mcp# or without Docker:  latent-gate mcp --http --port 8000

Serves streamable HTTP at /mcp (stateless, so any instance can answer any request) and /health. Because it's remote, it also works where local MCP servers can't run: claude.ai on the web and mobile, and any client that only takes a connector URL. The main tool there is fetch_url_optimized. It fetches a page, JSON API or raw log by URL and returns the visible text, compacted and deduplicated without a summarizing model. Addresses that aren't public (localhost, 10.x, cloud metadata) are refused, and that's checked on the live connection, so DNS tricks don't get around it. In this mode:

• read_file_optimized, compress_image and get_stats are hidden. The first two would read the server's disk, and stats would mix all callers' usage.
• compress_* tools are listed only when LATENTGATE_HOSTED_OLLAMA=1 (the host runs Ollama).
• Each call's input is capped at 2MB.
• With LATENTGATE_API_KEY set, requests need Authorization: Bearer <key>. Give the key to your gateway and nobody else, so callers can't bypass the paid plan.

Speed Optimizations

Cost Benchmarks

Image Queries (by provider, estimated)

Estimates from each provider's published image-token formulas versus a ~150-token local description.

Text: measured, reproducible

Deterministic optimizer on built-in realistic inputs, counted with tiktoken (o200k_base). Facts kept = share of numbers, identifiers, URLs, file names and code preserved verbatim. Reproduce with latent-gate --optimizer-benchmark (no Ollama or API key needed).

Logs lose individual ids/timestamps when folded (the fold keeps first, last and value ranges), and RAG drops facts from documents irrelevant to the question — both by design.

How it works (safest stage first; see latent_gate/optimizer.py):

1. Protect code blocks, inline code, URLs and quoted strings — restored byte-for-byte
2. Lossless: whitespace/Unicode cleanup, JSON minification (values untouched), duplicate folding
3. Log folding: runs of log lines differing only in numbers/ids → first, last, and ranges
4. Filler: pure pleasantries ("Hi!", "Thanks in advance!") and hedging phrases removed
5. Selection (only over a budget, or aggressive): BM25 question-relevance + requirement cues, original order preserved, the user's actual ask is never dropped

Guarantees: output never has more tokens than input; same input → same output (so provider prompt caching keeps working); when a local LLM rewrite is used it must be smaller and keep every fact, otherwise the deterministic result is sent.

from latent_gate import optimize
r = optimize(long_prompt, question="What failed?", level="balanced", max_tokens=2000)print(r.optimized_tokens, r.savings_pct, r.stages)

Video

Selective decoding skips remote calls for frames similar to the previous one (~2.85x fewer calls on typical footage).

At Scale (10,000 image queries with gpt-4o-mini, estimated)

Cost Tracking

from latent_gate import CostTracker
tracker = CostTracker()tracker.record_usage(    query_type="image",    provider="openai",    model="gpt-4o-mini",    input_tokens=150,    output_tokens=200,    tokens_saved=1000,    compression_ratio=6.7,    latency_ms=1500,)
# Session statisticsstats = tracker.get_session_statistics()print(f"Total cost: ${stats['total_cost']:.4f}")print(f"Tokens saved: {stats['total_tokens_saved']}")
# Cost projectionprojection = tracker.get_cost_projection(    daily_queries=1000,    provider="openai",    model="gpt-4o-mini")print(f"Monthly savings: ${projection['savings']['monthly']:.2f}")
# Export reporttracker.export_report("usage_report.json", fmt="json")tracker.export_report("usage_report.csv", fmt="csv")

Multi-Language Support

from latent_gate import detect_language, MultiLanguageProcessor
# Detect languagelang = detect_language("Esto es un texto en español")print(f"Detected: {lang.name} ({lang.confidence:.0%})")
# Process with auto-translation to Englishprocessor = MultiLanguageProcessor()text, lang_info = processor.process("Texto en español para analizar")print(f"Language: {lang_info.name}, Translated: {text[:100]}...")

Project Structure

latent-gate/├── latent_gate/│   ├── __init__.py           # Package exports and version│   ├── config.py             # PipelineConfig dataclass│   ├── config_loader.py      # YAML/TOML/JSON config loading│   ├── payload.py            # SemanticPayload (compact representation)│   ├── text_processor.py     # TextPayload + TextProcessor (local compression)│   ├── local_processor.py    # X-Encoder + Predictor (Ollama vision pipeline)│   ├── remote_decoder.py     # Y-Decoder (OpenAI, Anthropic, Google, Ollama)│   ├── selective_decoder.py  # Cosine/Jaccard similarity for skip decisions│   ├── fast_client.py        # Connection pooling + model preloading│   ├── cache.py              # Content-hash disk cache│   ├── pipeline.py           # LatentGatePipeline (main orchestrator)│   ├── async_pipeline.py     # AsyncLatentGatePipeline│   ├── video_processor.py    # Video frame extraction + batch processing│   ├── cost_tracker.py       # SQLite-based cost analytics│   ├── mcp_server.py         # MCP server (Model Context Protocol)│   ├── api_server.py         # FastAPI REST server│   ├── cli.py                # Command-line interface│   ├── logging_config.py     # Structured logging with rotation│   ├── plugin_system.py      # Custom processor plugins│   └── multilang.py          # Multi-language detection and translation├── integrations/│   ├── agent_skills/         # Prompt compression skill + scripts│   ├── vscode-extension/     # VS Code extension source│   ├── cursor/               # Cursor rules and MCP config│   ├── continue_dev/         # Continue.dev config│   ├── langchain/            # LangChain integration wrapper│   ├── llamaindex/           # LlamaIndex retriever integration│   └── openai_functions/     # OpenAI/Anthropic function schemas├── tests/                    # 240+ tests (unit + integration)├── website/                  # React-based analytics dashboard & landing page├── deployments/              # Kubernetes Helm configs├── .github/workflows/        # CI + publish workflows├── Dockerfile├── docker-compose.yml├── pyproject.toml└── requirements.txt

Community

• GitHub Discussions — Feature requests, Q&A, showcases

Contributing

Contributions welcome! See CONTRIBUTING.md.

Development Setup

git clone https://github.com/KathanModh259/latent-gate.gitcd latent-gatepython -m venv .venvsource .venv/bin/activate       # Linux/macOS.venv\Scripts\Activate.ps1      # Windows
pip install -e ".[dev]"

Run Tests

pytest tests/ -v

Priority Areas

• Additional vision model support (Florence-2, InternVL, Qwen-VL)
• Custom similarity plugins for domain-specific use cases
• WebSocket support for real-time streaming
• Advanced cost analytics and optimization suggestions
• Plugin development for specialized industries
• Test coverage improvements
• Documentation and examples

Citation

@software{latentgate2026,  author  = {Kathan Modh},  title   = {LatentGate: Local-First Semantic Compression Pipeline},  year    = {2026},  url     = {https://github.com/KathanModh259/latent-gate},  version = {1.3.0}}

Inspired by VL-JEPA (Meta FAIR, 2025).

License

Custom Proprietary License — see LICENSE.