llmtrim

[llmtrim]

llmtrim

Local proxy that compresses LLM API traffic so you pay less. Same answers, smaller bill.

_{−31% input · −74% output · −66% round-trip cost · 112 live A/B cases · ~5 ms/call · no model to load}

_{Using Claude Code? One install also gets you /sub to serve it through CLIProxyAPI.}

_{Proxy · CLI · MCP · library (Python · Ruby · Swift · Kotlin · JS/WASM)}

[llmtrim status: Overview savings dashboard, Sessions, Detail, and Sub routing tab (CLIProxyAPI presets, fallback hops, and model map)]

[CI] [License: MPL 2.0] [crates.io] [npm] [npm downloads] [Rust 1.88+]

What it does • Install • Day to day • In action • Works with • Claude Code • Numbers • Config • CLI & library

What it does

You run Claude Code, Codex, Cursor, or your own app. Every turn, the tool sends a large request: system prompt, tools, history, raw command output. You pay for every token of that, including the parts that do not help the model.

A 200-line build log with two errors. Tool schemas resent on every call. JSON with hundreds of near-identical rows. That bulk is still billed.

llmtrim sits on your machine as a local proxy, trims the waste, and forwards a smaller request. The reply is unchanged. You keep the same tools and answers; you spend less.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …                    ◀──────── reply ──────────
  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …                            (on your machine)                    ◀──────── reply ──────────  (same answer)

Compression cannot raise your bill or break a request; worst case is zero savings. Everything runs locally, nothing is sent to us. In action →

For Claude Code the same install also wires /sub to route sessions through another subscription. Details →

Get started

npm install -g @llmtrim/cli@latest && llmtrim setup# open a new terminal, then keep workingllmtrim status

That's it. setup starts a local proxy and wires your shell. When Claude Code is present, it also turns on /sub. You do not run a separate install for that.

Any tool that honors HTTPS_PROXY works (Claude Code, Codex, Cursor, Aider, your SDK). GitHub Copilot does not (certificate pinning). Full list →

# Linux / macOScurl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh
# Windows (PowerShell)irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex
# Package managersbrew install fkiene/tap/llmtrimcargo binstall llmtrimscoop install llmtrimdocker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim

llmtrim caopenssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"

Day to day

llmtrim status     # savings + health  (aliases: monitor, gain)llmtrim update     # new release, restart daemon, refresh integrationsllmtrim ensure     # match the recommended install state on this machine

After setup, update, or ensure, owned Claude Code /sub stays in sync with the binary.

Time series: llmtrim status --daily · --weekly · --monthly · --json · --csv.

In action

An agent ran a build. The tool returned 58 lines; two were errors. All 58 would have been billed.

4,662 chars → 978 (−79%). Errors stay verbatim. Repeated INFO lines fold into a template plus the values (lossless when the range is regular).

# before (noise + signal)[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)… 28 more near-identical INFO lines …[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`… 25 more INFO lines …[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once[2026-06-13T10:03:02Z] INFO  build failed, 2 errors
# after (errors verbatim; INFO folded losslessly)[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)][2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24][2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Try it on a request body of your own:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log folding is one stage. Others kick in on different waste:

[!IMPORTANT] Compression cannot raise your bill or break a request. Each stage is re-measured with the provider's real tokenizer and undone if it does not save tokens. If the provider rejects the compressed body, the original is resent. Worst case is zero savings.

Existing prompt-cache prefixes (cache_control) are left alone. First-arrival recoverable shaping (llmtrim recall r_…) is off by default; live-zone tool-output windowing still runs, and a re-run of the same tool ships in full. To skip shaping for a command (verbatim stdout, including a terminal trailer), set toolout_passthrough = ["*gpt.sh*"] or prefix the command with LLMTRIM_TOOL_OUTPUT=passthrough. Lines starting with LLMTRIM_KEEP: survive windowing even when the rest is clipped.

Claude Code

When ~/.claude exists, setup, update, and ensure wire /sub (and routed subagents). No separate install command for that.

llmtrim sub on                  # install + start CLIProxyAPI, enable redirectllmtrim sub on grok             # write opus/sonnet/haiku/fable → that CLI's modelsllmtrim sub auth                # CLIProxyAPI TUI — sign inllmtrim sub models              # official + live sidecar modelsllmtrim sub map on opus grok-4.6llmtrim sub chain anthropic,codexllmtrim sub mode fallback       # try hops in order when the current one failsllmtrim sub statusllmtrim sub off

llmtrim agents install          # also installed/refreshed by setup, update, and ensure

/sub on [optional:cli-or-model]  # gemini, codex, claude, grok, kimi, vertex, qwen, copilot, or a model id/sub off/sub status

llmtrim sub anthropic-login keep   # connectors OK; Anthropic login requiredllmtrim sub anthropic-login skip   # default: no Anthropic /login; connectors off

Use it as a CLI, MCP, or library

Same engine, no proxy required. No extra model calls; compress runs in-process.

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.jsonecho '{"model":"gpt-4o","messages":[...]}' | llmtrim send --provider openai

use llmtrim_core::{compress, ir::ProviderKind};let out = compress(request_json, Some(ProviderKind::OpenAi))?;

import llmtrimout = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")

import { compress } from "@llmtrim/js";const out = compress(requestJson, "openai", "aggressive");

llmtrim mcp install          # Claude Codellmtrim mcp install --print  # paste into any client

{  "mcpServers": {    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }  }}

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA:

No proxy: any MCP client can call llmtrim as tools (llmtrim mcp install), or use the CLI / library.

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Non-LLM connections pass through untouched.

Configuration

Default is fine for most traffic. auto inspects each request and picks compressors by shape (tools → agent, code → code, long Q&A → rag, else aggressive).

Override with LLMTRIM_PRESET=<name> or preset = "<name>" in $XDG_CONFIG_HOME/llmtrim/config.toml:

Claude Code options (subscription reroute) are under Claude Code.

export LLMTRIM_UPSTREAM_PROXY=http://host:port# or with auth: http://user:pass@host:port  (redacted in logs)

The numbers

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

[llmtrim cuts the round-trip bill on both ends: $0.0365 original vs $0.0126 compressed, −66% cost, across 112 live A/B cases]

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

make -C crates/llmtrim-cli/bench data(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \   --corpus bench/data/squad2.jsonl --preset rag \   --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Methodology, per-corpus frontier, and confidence intervals: crates/llmtrim-cli/bench/README.md. Reproduce it:

make -C crates/llmtrim-cli/bench data   # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite)  # live A/B across all corpora (needs OPENROUTER_API_KEY)(cd crates/llmtrim-cli/bench/scripts && PYTHONPATH=. python3 -m benchkit.tools.chart)  # regenerate the chart + table

How it compares

Each tool compresses one slice of the request. llmtrim compresses input and output, leaves the cached prefix untouched to keep the prompt cache stable, and scores on whether the answer survives the cut, not on tokens removed. Both axes below use the o200k_base encoder and reproduce from this repo.

Input

Input reduction (deterministic) next to answer quality from a live A/B. Quality is the drop vs llmtrim at each tool's compared setting (✅ held, a statistical tie; ❌ significantly lower), so a big reduction with a ❌ means the tool bought tokens by losing answers:

Overhead is the median per-call compress time (Python wall-clock, not like-for-like CPU): Headroom and leanctx run ML on CPU here (faster on a GPU) and pay a one-time model load on top (~3 s and ~4 s); llmtrim is Rust and entroly is lexical, so neither does.

• auto is the quality-gated default; aggressive accepts lossy cuts where the gate holds.
• Headroom drops to 0% with its ML disabled (its routers no-op on prose).
• leanctx and entroly are lossy with no quality gate; entroly has no low-reduction mode.

Headroom ties at matched reduction (24-25%, n=30, not significant) but its longer answers hit the model's output-token limit and get truncated 12 times to llmtrim's 2, the output inflation behind its higher cost. leanctx (measured at 26%) and entroly (at 69%, its mildest) score significantly lower than llmtrim (n=20), and fall further at their headline reductions (vs-leanctx, vs-entroly).

Output

Output reduction by asking for terser responses, on a paid live call over 9 coding prompts:

The cost is the 949-token system prompt caveman resends on every request (right column); llmtrim's is 19 for nearly the same cut. Both still net-save here, so caveman's deeper cut comes out ahead only when the output it removes is worth more than the 949 tokens it adds back (vs-caveman artifact).

The tools stack: RTK shrinks CLI output, then llmtrim compresses the tool schemas on top. Full head-to-heads: crates/llmtrim-cli/bench/README.md.

Known limits

These are surfaced by the same A/B that proves the savings:

• Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.
• Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.
• The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.
• "Lossless" is input-side, not response restoration. A lossless stage preserves the information the model reads (a folded log run, a TOON-encoded array, an abbreviation legend the model decodes in-prompt), and the token gate reverts any input cut that doesn't pay off. The engine does not transform the model's response back to an original form.

Acknowledgments

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Found a problem?

llmtrim doctor          # diagnosellmtrim doctor --fix     # diagnose + apply repairsllmtrim ensure          # same repair path

Each failing check names its fix. If a request was mangled, set LLMTRIM_CAPTURE_DIR and open an issue with the before/after pair.

If llmtrim saved you money, a ⭐ helps others find it.

Star History

_{Licensed under MPL-2.0. Use llmtrim freely in your stack, including commercially, with no source-disclosure obligation for your own code; the file-level copyleft applies only to modifications you make to llmtrim's own source files. Contributions via DCO sign-off.}