io.github.TMHSDigitalPublished by GitHub account TMHSDigitalv0.6.1Updated Nov 4, 2026
Run, fit, and route work to local LLMs (Ollama, LM Studio, llama.cpp, MLX, vLLM, and more).
Local AI's local MCP server; no sign-in required; STDIO; 19 tools.
No AI overview yet: these facts come from the registry entry and the server itself.
Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.
Follow the launch instructions in the repository.
One MCP server for every local model runtime on your machine. It knows what fits your GPU, and it never sends a prompt to the cloud unless you tell it to.
[npm] [CI] [License: CC-BY-NC-ND-4.0] [Type]
[Install in Cursor] [Install in VS Code] [Claude Desktop extension]
Local AI MCP gives your agent (Claude, Cursor, VS Code, and others) one control plane for Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM and Foundry Local. Your agent can see what's installed and loaded, check what will actually fit in your VRAM, get models ready, and hand work off to the best local model with a single delegate call.
fit_check sizes the KV cache from the model's actual layer architecture, adds up VRAM across GPUs, models partial GPU/CPU offload, and understands Apple Silicon unified memory. It answers "will this model run, and where?" with numbers rather than guesses.provider argument never uses one. Every result reports provider, host and local, so you can see where inference ran.suggest_model, then ensure_modellist_loaded, then unload_model (or use the free-vram prompt)benchmark { models: [...] }search_models { source: "huggingface" }delegate { task: "extract", responseFormat: { type: "json_schema", ... } }fit_check { model, contextLength: 32768 }Install at least one runtime (Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM or Foundry Local), then add the server to your client. No configuration is needed for runtimes on their default ports.
Claude Code
Claude Desktop. Download the .mcpb file from the latest release and open it. You get a settings screen for hosts and no JSON to edit. Alternatively, add this to claude_desktop_config.json:
Cursor / VS Code. Use the install buttons above, or add the same command/args to .cursor/mcp.json or .vscode/mcp.json (VS Code uses a top-level "servers" key instead of "mcpServers").
Windsurf, Cline, Zed and other clients. Any stdio MCP client works: the command is npx, the args are ["-y", "@tmhs/local-ai-mcp"].
Windows: a few clients can't launch
npxdirectly. If the server doesn't start, use"command": "cmd", "args": ["/c", "npx", "-y", "@tmhs/local-ai-mcp"].
Adding a hosted provider (optional). Put MOONSHOT_API_KEY in the server's env. It is only used when a tool call names provider: "moonshot", unless you also set LOCAL_AI_ALLOW_HOSTED_FALLBACK=1.
You can run more than one instance of a runtime, for example a second Ollama box on your LAN, with LOCAL_AI_PROVIDERS='[{"type":"ollama","name":"ollama-gpu2","host":"http://192.168.1.20:11434"}]'. Use LOCAL_AI_DISABLE=lmstudio,dockermodelrunner to skip probing runtimes you don't use.
*.local, Tailscale CGNAT and host.docker.internal count as local. You can override it per instance in LOCAL_AI_PROVIDERS.provider argument, complete, embed, delegate, benchmark, load_model and the other model-scoped tools only consider live local providers that have the model. If the model exists only on a hosted provider, the call fails and tells you how to opt in.health_check contacts them.search_models with source: "huggingface" queries the Hugging Face Hub, and only when you ask it to. Set LOCAL_AI_DISABLE_NETWORK_SEARCH=1 to forbid it.Every tool declares MCP annotations (read-only, destructive, open-world) and an output schema, and returns structured content.
Resources: local-ai://system, local-ai://loaded (subscribable; updates when models load or unload), and local-ai://models/{provider}.
Prompts: setup, free-vram, benchmark-my-models, pick-model.
Every setting is an environment variable, and blank means the default. See .env.example for the full annotated list.
health_check. Check the host and port. From WSL, use the Windows host IP. From a container, use host.docker.internal. For Docker Model Runner, enable host-side TCP support.lms CLI as described in the LM Studio CLI docs so it is on PATH, then restart your MCP client.embed fails. Start llama-server with --embeddings and an embedding-capable model.contextLength (it sets num_ctx). The result includes a warning when truncation is likely.list_models) or only available on a hosted provider. Name that provider explicitly to use it.system_resources reports available memory (free, inactive and purgeable pages) and the Metal working-set limit, not total RAM.CI runs on Linux, Windows and macOS with Node 22 and 24, plus a Node 20 runtime check. A nightly workflow tests the adapters against real Ollama and llama.cpp. See CONTRIBUTING.md, including Adding a runtime adapter.
Questions and ideas go in Discussions. Report bugs with the issue templates, and security problems through private reporting. If this saves you time, consider sponsoring.
CC-BY-NC-ND-4.0. See LICENSE. Contributions are accepted under the DCO and the inbound grant described in CONTRIBUTING.md.
Built by TMHSDigital
Source: README.md at commit 5ea6f54
List providers
List configured model runtimes (Ollama, LM Studio, llama.cpp, Docker Model Runner, and any opt-in MLX / vLLM / Foundry Local / OpenAI-compatible / hosted instances) with host, kind (local or hosted), live status, and capabilities.
providerList installed models
List models installed/available on each live provider, keyed by provider. Pass provider to query just one.
providerList loaded models
List models currently loaded into memory on each live provider, with VRAM use and context length where the runtime reports them.
providerModel info
Detailed metadata for one model (family, parameter size, quantization, context length, and layer architecture where available). Without provider, queries the live local providers that have the model.
modelrequiredproviderPull model
HEAVY: Download a model onto ONE provider (multiple GB, can take a long time). Streams download progress via MCP progress notifications when the client sends a progressToken. Without provider, uses the provider that already has the model, else the first live local provider that can pull.
modelrequiredproviderRemove model
DESTRUCTIVE: Permanently delete a model from one provider. provider is REQUIRED (no fan-out). If the client supports elicitation the user is asked to confirm directly; otherwise confirm:true is required.
modelrequiredconfirmproviderrequiredLoad model
Load a model into memory on ONE provider so it is ready for inference. Optional keepAlive (e.g. '5m', '1h') and contextLength. Without provider, uses the live local provider that has the model.
modelrequiredproviderkeepAlivecontextLengthUnload model
Unload a model from memory to free VRAM/RAM on ONE provider. Without provider, uses the live local provider that has it loaded.
modelrequiredproviderEnsure model is ready
Make a model ready in one call: optionally pull it, check it fits current free VRAM, unload other models if needed (evict: 'lru' or 'all'), load it, and verify. Use dryRun:true to get the plan without changing anything.
evictmodelrequireddryRunproviderallowPullkeepAlivekvCacheTypecontextLengthHealth check
Check whether each provider is reachable and report its version (and live metrics such as vLLM KV-cache usage). Without provider, checks every configured provider, including hosted ones.
providerSystem resources
Report local hardware: platform, total/available RAM, CPU count, and GPUs with total/free VRAM (NVIDIA via nvidia-smi, AMD/Intel via sysfs or the Windows registry, Apple Silicon unified memory). Each GPU names its data source.
Takes no parameters.
Fit check
Will a model run on this machine, and where? Uses the installed model's size and exact layer architecture when the runtime exposes it (else parameter-count or catalog estimates) to size weights + KV cache for a context length, then compares with free VRAM summed across GPUs and available RAM. Returns target gpu / partial (GPU+CPU split) / cpu / none, with the numbers and how the KV estimate was made. Credits back memory the model already uses if it is loaded.
modelrequiredprovidersizeByteskvCacheTypecontextLengthparameterSizeBenchmark
HEAVY: Runs REAL inference. Measures a model on this machine with a warmup (absorbs load time) plus N streamed runs: time-to-first-token, generation tokens/sec (runtime-reported when available, else measured after the first token), and prompt-processing speed. Pass models[] to compare several models with the same prompt, ranked by generation speed.
runsmodelmodelspromptwarmupprovidermaxTokensSearch catalog
Search the built-in curated catalog of well-known local models (Ollama tags) by name, family, or task. Offline and instant. For live results with per-quant fit, use search_models with source 'huggingface'.
queryrequiredSuggest model
Recommend catalog models for a task, ranked for THIS machine: fully-on-GPU first, then partial GPU offload, then CPU-only. Within a tier the largest (most capable) model comes first unless preferSpeed is true. Flags models already installed on a live provider and explains each pick.
taskrequiredkvCacheTypepreferSpeedcontextLengthSearch models (live)
Find models to download, annotated with whether each fits this machine. source 'huggingface' searches GGUF repos on the Hugging Face Hub and returns per-quant sizes, the best quant for this hardware, and pullable references (Ollama: hf.co/<repo>:<quant>). source 'catalog' searches the offline curated list. The Ollama library has no public search API, so it is not a source.
tasklimitqueryrequiredsourcefitsOnlycontextLengthDelegated completion
DELEGATED INFERENCE: Offload a completion to a model you choose for cost savings and privacy. Without provider, runs on the live LOCAL provider that has the model (never a hosted API unless LOCAL_AI_ALLOW_HOSTED_FALLBACK is set); the result's 'provider', 'host' and 'local' fields say where it ran. Supports system prompts, JSON / JSON-Schema output (validated 'parsed' field), seed/topP/topK, image inputs for vision models, and contextLength for long inputs. Streams tokens via MCP progress notifications when the client sends a progressToken.
seedstoptopKtopPmodelrequiredpromptstreamsystemoptionsmessagesprovidermaxTokens+4 moreDelegated embeddings
DELEGATED EMBEDDINGS: Generate embeddings for a string or an array of texts. Without provider, runs on the live local provider that has the model. Always returns one vector per input.
inputrequiredmodelrequiredproviderDelegate a task to the best local model
ONE CALL to offload work locally: pick the best model for a task on THIS machine, make it ready, and run it. Prefers models already loaded (no load cost), then installed models that fit your free VRAM, then (with allowPull) the best catalog model that fits. Task specialists (coder, vision, reasoning) win over general models; then larger fitting models win, grouped into size tiers (within ~20% parameter count); inside a tier, the model benchmarked fastest on this machine wins (run benchmark to record speeds; results expire after 14 days). Never uses a hosted provider unless allowHosted is true. Returns the completion plus a 'routing' block explaining the choice.
seedstoptaskrequiredtopKtopPpromptstreamsystemoptionsmessagesproviderallowPull+7 more80% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.
Declare a license in the repository or the registry entry.
# illustrative sessionYou: Summarise these 40 pages of meeting notes locally, I don't want them in the cloud.Agent: → delegate { task: "summarize", prompt: "...", contextLength: 32768 } ← routing: ollama / qwen3:14b — already loaded (no load cost); general model; fits: gpu local: true · 412 tokens · 38 tok/sclaude mcp add local-ai -- npx -y @tmhs/local-ai-mcp{ "mcpServers": { "local-ai": { "command": "npx", "args": ["-y", "@tmhs/local-ai-mcp"] } }}npm installnpm run build # tsc -> dist/npm test # vitest, fully offline (mocked HTTP, stubbed hardware)npm run lint # Biomenpm run typecheck # src and testsnpm run gen:tools # regenerate mcp-tools.json after changing toolsnpm run smoke # boot the built server over stdio