Sezzlee Local LLM

@sezzlee/llm-mcp

MCP server that lets a planning agent (Codex, Claude Code, Cursor) hand off bounded language work to a local model. It builds on @sezzlee/mcp-core and names no other @sezzlee/* package.

Quick start

{  "mcpServers": {    "local": {      "command": "npx",      "args": ["-y", "@sezzlee/llm-mcp"],      "env": {        "SEZZLEE_LLM_BASE_URL": "http://127.0.0.1:11434",        "SEZZLEE_LLM_MODEL": "qwen3:8b"      }    }  }}

In Codex, the same block goes under mcp_servers.local. If it runs with approval_policy = "never", default_tools_approval_mode = "auto" is also required, or Codex refuses the call.

Tools

Three tools. Two are read-only; local_map only adds new files to the server's own output directory.

Configuration

SEZZLEE_LLM_MODEL=qwen3:8bSEZZLEE_LLM_ROOT=.                            # default: the working directorySEZZLEE_LLM_OUTPUT_DIR=.llm-mcp/out           # default; relative to root, must stay inside itSEZZLEE_LLM_BASE_URL=http://127.0.0.1:11434   # defaultSEZZLEE_LLM_NUM_CTX=16384                     # default; at least 4096SEZZLEE_LLM_KEEP_ALIVE=30m                    # defaultSEZZLEE_LLM_TIMEOUT_MS=300000                 # default; one request
npx -y @sezzlee/llm-mcp

Only the model name is required. Parsing is fatal, reaching the host is not: the server still starts and answers tools/list with the host down, and local_status reports reachable: false. The model is warmed up right after the server starts serving.

Set SEZZLEE_LLM_NUM_CTX to what the GPU actually delivers. Ollama does not refuse an oversized input — it silently drops the head — and the input budget is derived from this value.

Limits

Rules

• Three lint-enforced layers, three root entrypoints. src splits into platform/ (the mcp-core and Node boundary), backend/ (the model host), and tools/ (the MCP surface). tools/ sees the host only through backend/port.ts; the concrete backend is wired in cli.ts.
• Only src/backend/ollama.ts reaches the network, and only through the global fetch. That global and the node:http/net/tls family are lint-banned everywhere else.
• process.env is read only in cli.ts, each variable accessed by name.
• File paths are resolved against the workspace. The requested path and its realpath are checked separately, so a symlink pointing outside the workspace is refused too. readText accepts only a WorkspacePath-branded path that has passed both checks. In error envelopes, the workspace's absolute path appears as ..
• The server counts tokens, not the model. An input over num_ctx × 0.45 never reaches the host; an input that could not fit under any circumstances is refused before it is even read. The answer is capped at num_ctx × 0.4.
• Only src/platform/workspace.ts touches the filesystem, lint-enforced. The single writer is createOutput: it re-checks the output directory's realpath, the server itself picks the file name, lowers it to a safe alphabet and fixes the extension to .csv, and opens it wx — so it never overwrites an existing file or a symlink, and nothing is ever deleted (mcp-core).
• local_map carries the row number to the model and back. An id-less format lost the row order. A skipped row is asked again once; if it is still missing, it is left unlabeled and counted — never guessed.
• A chunked input is not truncated either. Chunks split on paragraph, then line, boundaries; each chunk yields notes, the notes are merged and the instruction applied one last time. If the merged notes still do not fit the budget, the merge round repeats (up to 3); if that still fails, the call is refused as input_too_large.
• One request per host at a time. createSerialBackend queues requests: the GPU does not run them in parallel, while Codex fires calls concurrently. probe bypasses the queue.
• A new backend kind is just another implementation of the Backend interface. The rest of the server does not change.

Design notes

The measurements below are what the rules above are built on.

Why an MCP server instead of pointing Codex at Ollama directly

Four ways of getting Codex to a local model were tried; only the MCP server worked.

The server's working directory is Codex's workspace: every file argument the tools accept is resolved against it, and a path that resolves outside it is refused.

Model choice

gpt-oss:20b was measured scoring 0/100 on the id-bearing JSON schema local_map and structured local_task calls depend on — the model could not hold the schema. A qwen3 model (qwen3:8b in the examples above) is what the measurements in this repository were run with; SEZZLEE_LLM_MODEL has no built-in default, so it must be set explicitly.

Codex alone on tabular data

On a 400-row labelling task with clear cues, Codex working alone was as accurate as delegating (14/14 correct) and about 3.5x faster: 48 s versus 166 s through local_map. This is why DELEGATION_INSTRUCTIONS (products/chat/api/src/codex/delegation-instructions.ts) allows Codex to write a script instead of delegating when the data is tabular with clear patterns — delegating only pays off on long or unstructured text, not on rows a script can already parse.

Rejected alternatives

• The name ollama-mcp. Too narrow once the backend became pluggable; llm-mcp follows the repo's convention of naming a server after the resource behind it, not the current backend.
• A private worker inside a consumer product. The server has nothing chat-specific in it — tools take a file path and an instruction, schemas are plain JSON Schema on the MCP surface. Keeping it inside one product would close it to outside contribution and force reinventing the payload budget that @sezzlee/mcp-core already provides.
• Using an existing open-source repo. Most of the ones reviewed are Python, and one carries a license that forbids modification; none of them know Codex's approval mode, its workspace boundary, or the payload budget. Ideas were borrowed where they held up: per-kind ready-made prompts (from ollama-handoff), file-path-taking tools and failover (from LocalTokens), and "the worker produces bounded output, the strong side accepts it" (from Shahriar/ollama-mcp-server).
• One tool per job kind. Every extra tool schema is token cost on every Codex turn; one local_task tool with a kind enum covers the same ground for less.
• Raising concurrency instead of queueing. Four parallel requests to one GPU were only 1.1x faster than four sequential ones — the GPU does not run requests in parallel, so a queue is the only thing that keeps a call inside its own timeout.

Out of scope (deliberately)

• An agent loop on the local model. A reviewed repo's small model, run through a full agent harness, took 5–7 minutes on a simple task and once reported having done work it had not done. Agent work already has Codex's own spawn_agent.
• Embedding. bge-m3 is available on the host but unused; add it if a search need appears.
• Reading .xlsx directly. v1 reads CSV and plain text only. An .xlsx input goes through @sezzlee/excel-mcp from the calling side; teaching this server the format directly is out of scope for now.

Measured runs

All Codex runs below used gpt-5.6-luna at low reasoning effort, against Ollama 0.34.2 with num_ctx 16,384. "Cache-free" is input_tokens − cached_input_tokens.

Development

pnpm turbo run test --filter=@sezzlee/llm-mcp

No local model is needed for this — fetch is faked.

test/live.spec.ts runs against a real Ollama instance:

SEZZLEE_LLM_LIVE=1 SEZZLEE_LLM_BASE_URL=http://127.0.0.1:11434 SEZZLEE_LLM_MODEL=qwen3:8b \  pnpm turbo run test --filter=@sezzlee/llm-mcp -- test/live.spec.ts