Local Llm Worker

Reading a 5,000-line test log or three docs pages costs the same frontier-model tokens as hard architectural work. local-llm-worker is an MCP server and Claude Code plugin that moves that bulk work onto the GPU (or CPU) you already own:

• offload: your local model runs the noisy command or reads the big file. Claude gets the answer.
• research: your local model searches the web, reads the pages in full and returns a cited answer. Claude never sees the pages.
• delegate (opt-in): your local model writes one file in an isolated git worktree and retries until a test Claude wrote first passes. Claude gets a one-line verdict, not the code.

> offload  npm test 2>&1   "Which test fails, and why?"`fixed coupon never goes below zero` (test/cart.test.js:404): applyCoupon(5, {type:'fixed',value:10}) returns -5, expected 0.— qwen3-coder:30b-a3b-q4_K_M · 19,132 input tokens read locally

Why

• Any model, any hardware. No hardcoded models, no GPU assumptions. CPU-only works, just slower.
• Parallel by default. Several calls in one turn, or from several subagents, run at once. The concurrency limit is a config value, not a hardcoded lock.
• Fits the context automatically. Material is sized to the model's context window per text (number-heavy logs need more tokens than prose). Anything cut is reported.
• Zero dependencies. Plain Node ≥ 18. Installing from git needs no npm install.
• Agent-configurable. One configure call shows the config, the backend and its models, and changes any setting. It takes effect on the next call, with no restart.

Install

Prerequisites: Node ≥ 18, git, and a running local model server (e.g. ollama pull qwen3-coder:30b).

Claude Code plugin

/plugin marketplace add JoblessJoe/local-llm-worker/plugin install local-llm-worker@local-llm-worker

Then just ask: "set up local-llm-worker for my machine". Claude finds your backend, picks a model, and asks which tools it should use on its own (auto_use). By default that's offload and research; delegate is opt-in. Every tool also works whenever you ask for it.

Make Claude use it every time

A session-start reminder nudges Claude toward the tools you chose. For dependable use, add this to your CLAUDE.md; setup offers to do it for you:

## Local LLM (local-llm-worker)- Run test suites, builds and other noisy commands through `offload` (`command`), and read logs or files over ~300 lines through it, instead of reading the output yourself.- Use `research` for web lookups instead of WebSearch/WebFetch.

With this in place, Claude ran a noisy failing test suite through offload every time we tried. That session cost 40 % less than one that read the output itself.

Use it in any MCP client

Claude Desktop, Cursor, Windsurf, and others:

{  "mcpServers": {    "local-llm-worker": {      "command": "node",      "args": ["/path/to/local-llm-worker/src/index.js"],      "env": { "LLW_MODEL": "qwen3-coder:30b" }    }  }}

Claude Code without the plugin: claude mcp add local-llm-worker -- node /path/to/local-llm-worker/src/index.js

How delegate works

flowchart LR    A[Claude writes the test<br/>+ spec + invariants] --> B[delegate]    B --> C[fresh git worktree<br/>+ your uncommitted changes]    C --> D[local LLM writes<br/>target file]    D --> E{run test}    E -- fail --> F[parse failures<br/>test source stays hidden]    F --> D    E -- pass --> G[copy file into checkout<br/>unless it changed meanwhile]    G --> H[Claude gets a one-line verdict]

• Isolated. Each call gets its own git worktree with your uncommitted changes mirrored in, plus symlinked node_modules / .venv, so parallel calls never collide.
• Locked scope. Exactly one target file, jailed to the repo, and never one of the test_files you name.
• Useful feedback. Retries get the parsed failing assertions (TAP, pytest, jest, vitest, go, cargo), not a stack-trace tail.
• Safe apply. If you or another delegate touched the target meanwhile, nothing is overwritten.
• Honest failure. After N attempts you get the last failure and the kept worktree. Transport errors are reported as errors, never as a model FAIL.
• Self-cleaning. Each delegate prunes llw-* worktrees left behind by a crashed server, and kept ones older than 24 h. Untracked files over 10 MB are not copied into the worktree (the verdict says how many were skipped).

The bundled skill teaches Claude how to write specs that pass: one invariant per test, exact API surface in context, properties instead of examples, and never delegating auth or money code.

Results

On one 24 GB GPU (Tesla P40) with qwen3-coder:30b-a3b-q4_K_M on Ollama:

Several calls run in parallel: three in one turn took as long as the slowest one.

Which model?

From a reproducible benchmark (bench/) of 120 delegate runs across six task types:

Retries matter: with failure feedback, qwen3-coder's pass rate rose by half from the first attempt to the third. offload and research work well with any of these. Every run is logged (see Stats), so you can measure your own setup.

Configuration

Agents should use configure. Humans can edit JSON. Layers (later wins):

1. built-in defaults
2. user: ~/.config/local-llm-worker/config.json (respects $XDG_CONFIG_HOME)
3. project: <git root>/.local-llm-worker.json
4. env: LLW_<KEY>, e.g. LLW_BASE_URL, LLW_MODEL. Integers are digits only, booleans true/false/1/0/yes/no, link_dirs a comma list or JSON array, headers a JSON object. An invalid value is ignored and configure lists it under warnings.

A config file with invalid JSON is an error that names the file.

// configure — no args: effective config, value sources, backend, model list// configure — change settings (null removes a key):{ "set": { "model": "qwen3-coder:30b", "num_ctx": 65536 }, "scope": "user" }

With no model set, it uses the backend's only model, or fails with the list. It never guesses, because the guess could be an embedding model.

Backends

Verified end-to-end on Ollama (GPU) and llama.cpp llama-server (CPU only). <think> blocks from reasoning models are stripped. If a backend reads less of the prompt than was sent, the result says so.

Tool reference

Stats

Every run appends one line to log_path:

{"ts":"2026-10-04T17:51:28.476Z","tool":"delegate","model":"qwen3-coder:30b-a3b-q4_K_M","api":"ollama","ok":true,"attempts":1,"in_tok":201,"out_tok":195,"ms":57909}

jq -s 'group_by(.tool) | map({tool: .[0].tool, runs: length, ok: (map(select(.ok)) | length), out_tok: (map(.out_tok) | add)})' ~/.local-llm-worker/runs.jsonl

FAQ

Does Claude ever see the generated code? Only if it asks (show_code: true) or reads the file. Review before merging anything that matters. A passing test is a gate, not a code review.

What should I not delegate? Auth, money, permissions, security, and anything whose spec has no single right answer. The skill tells Claude this too.

Can the local model cheat the test? It never sees the test source. It can write only the one target file, and that file can't be one of the test_files you list.

A long delegate or research call was cut off. The server sends MCP progress notifications every 15 s when the client asks for them (progressToken), but whether that helps depends on the client:

Does it send my code anywhere? Only to the base_url you configure, which is localhost by default. research sends your search query to your own SearXNG and downloads public pages. It never sends your files.

Development

npm test   # 88 tests, fake backend + fake web, no LLM or network needed

src/index.js is the stdio JSON-RPC server. src/worker.js holds the config, backend client, tools and failure extractor.

Releases are automatic: every push to main runs the tests, bumps the patch version everywhere it appears (npm run bump does the same locally; pass minor, major or x.y.z for more), tags it and creates a GitHub Release.

Roadmap

• more search backends (Brave, Tavily) alongside SearXNG
• multi-file delegates
• per-task model routing
• an async job API for clients with a 60 s tool limit (Claude Desktop, Cursor)

License

MIT © Johannes Tebbert