Gguf Mcp

io.github.arose26Published by GitHub account arose26v0.1.1Updated Oct 24, 2026

Inspect local GGUF/safetensors models: quantization, params, VRAM fit — headers only.

VerifiedSTDIODesktop onlyFiles & StorageAI & ML

Overview

AI-generated overview

Inspects local GGUF and safetensors model files, reporting architecture, quantization, parameter count, tensors, and VRAM fit.

What you can do
Reads only the headers of local GGUF and safetensors model files, so an assistant can answer questions about models on disk without loading tensor data. Tools include inspect_model for a one-call summary (format, architecture, parameters, quantization, context length, file size, tensor count), list_tensors for tensor names, shapes and storage types with filtering, estimate_vram for a fit check against a chosen context length, and get_metadata for the GGUF key-value store or safetensors metadata. It accepts .gguf files, .safetensors files, shard index JSON, model directories, and extension-less GGUF blobs detected by magic bytes.
When to use it
Useful when you keep local model files and want an assistant to identify them, compare quantization levels, list tensors, or check whether a model fits in a given GPU memory budget at a chosen context length. It suits local model management and planning rather than running inference.
Requirements
Runs locally over stdio, typically launched with npx gguf-mcp, so Node.js and an MCP client such as Claude Code, Claude Desktop, Cursor, or Windsurf are needed. Desktop only; no network access, accounts, API keys, or environment variables are declared.
Before you install
Read-only inspection of local files; the README states no network, API keys, or telemetry, and that files never leave the machine. VRAM estimates are modeled, not measured, and exclude some overhead. Tensor statistics, PyTorch .bin, ONNX, and remote HuggingFace queries are out of scope.

Installation

In SourceWeft

  1. Open Gguf Mcp in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

gguf-mcp

An MCP server that inspects local model files — GGUF and safetensors — so Claude and other LLMs can answer questions about the models on your disk:

  • "What is this .gguf? Architecture, quantization, parameter count?"
  • "Will this model fit in my 12 GB GPU at 8k context?"
  • "What tensors are inside, with what shapes?"
  • "Show me its chat template / RoPE settings / tokenizer config."

Headers only. The parser never touches tensor data, so inspecting a 70 GB model takes milliseconds and a few MiB of I/O. No network, no API keys, no telemetry — your files never leave your machine.

Quick start

Claude Code

bash
claude mcp add gguf -- npx -y gguf-mcp

Claude Desktop — add to claude_desktop_config.json:

json
{  "mcpServers": {    "gguf": {      "command": "npx",      "args": ["-y", "gguf-mcp"]    }  }}

The same npx invocation works in Cursor, Windsurf, and any other MCP client.

Tools

ToolWhat it does
inspect_modelOne-call summary: format, architecture, parameters, quantization, context length, file size, tensor count
list_tensorsTensor names, shapes, and storage types — filterable (attn, blk.0, ...)
estimate_vramFit check: exact weights size + modeled fp16 KV cache for your chosen context length
get_metadataThe GGUF key-value store (or safetensors __metadata__), filterable by key

Paths can be a .gguf file, a .safetensors file, a *.safetensors.index.json, or a model directory (sharded HuggingFace layouts are aggregated across shards). Extension-less GGUF blobs — like the ones in Ollama's ~/.ollama/models/blobs — are detected by magic bytes.

Design notes

  • Context-friendly by construction. A tokenizer vocabulary is 100k+ strings; metadata arrays are returned as {count, sample} summaries and long strings (chat templates) are truncated with a marker. The full data stays on disk where it belongs.
  • Honest estimates. estimate_vram reports exact on-disk weight bytes plus the standard KV-cache formula (2 × layers × context × KV heads × head dim × 2 bytes), and says what it excludes rather than faking precision.
  • Defensive parsing. Magic checks, version checks (incl. big-endian detection), truncation detection, and sanity caps on header sizes — malformed files produce specific, actionable errors.
  • Zero runtime dependencies beyond the MCP SDK and zod. The GGUF binary reader and safetensors parser are hand-rolled and unit-tested against synthetic files built in the test suite — no fixtures, no downloads.

Development

bash
npm installnpm test                 # offline unit tests (vitest) — synthetic model filesnpm run build            # tsc → dist/node scripts/smoke.mjs   # end-to-end: generates models, drives the server over stdio

Architecture: src/gguf.ts (binary header parser + VRAM math) and src/safetensors.ts (JSON header + shard index) are pure logic with no MCP imports; src/index.ts is the MCP wiring and path/format detection.

Out of scope

Tensor statistics (would require reading data), PyTorch .bin (pickle — unsafe by design), ONNX, and remote HuggingFace queries (HuggingFace has an official MCP server for that).

License

MIT

Source: README.md at commit e8f5468

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.1.1LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts