ModelSpec

io.github.turbobeestPublished by GitHub account turbobeestvb8da9336f3c5a0b56d47987251a29006a5b0487bUpdated Oct 24, 2026

Find AI models that fit a job from your requirements, sourced benchmarks and real cost.

The publisher's own server: model.tools — Zero-Auth Agent Utilities

VerifiedStreamable HTTPWeb executableData & AnalyticsAI & ML

Overview

AI-generated overview

Remote ModelSpec server that helps an assistant pick AI models matching a job's requirements, with sourced benchmarks, policy checks and hardware fit.

What you can do
  • Decide which models fit a stated job, returning ranked results, may_qualify entries, a decision_id and a snapshot
  • Compare 2 to 4 named models on sourced snapshot facts, offerings and evidence
  • Check a policy for licence, origin and data handling, returning pass, fail or undetermined per model or platform
  • Assess whether a workload fits given hardware, as measured, estimated or undetermined
  • Look up model cards, legacy use-case profiles and the decision vocabulary
  • Record a reliability rating for an answer after acting
When to use it
Useful when choosing between AI models for a specific workload and you want sourced, reproducible evidence rather than opinion, or when you need to check models against an internal policy or a hardware budget.
Requirements
Remote streamable HTTP endpoint; no local install. A ModelSpec bearer API key is required for all data tools, sent as an Authorization header (an x-api-key header is also accepted). Keys are obtained from the provider's pricing page. Claude Desktop setups use an npx bridge and need Node.js.
Before you install
Most tools consume paid credits per call, so usage costs money. The API key is a secret; the Desktop config example stores it in plain text in the config file. Feedback is keyless and forwards the caller's address for rate limiting, and its storage is described as off until covered by a privacy statement. Tools are read-only and do not change data.

Installation

In SourceWeft

  1. Open ModelSpec in the dashboard and add it to a workspace.
  2. Connect Authorization token credentials. They are stored privately and never shown on public pages.
  3. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.turbobeest%2Fmodelspec/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "modelspec": {
      "type": "http",
      "url": "https://api.modelspec.dev/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_TOKEN>"
      }
    }
  }
}

README

ModelSpec remote MCP server

Read-only MCP tools over the ModelSpec catalogue. Data tools require an API key. The site stays on Cloudflare Pages; this Worker only answers https://api.modelspec.dev/mcp and its public server card at https://api.modelspec.dev/.well-known/mcp/server-card.json.

Official registry listing

The root server.json describes the keyed remote server using the official registry's 2025-12-11 schema. Its checked-in version records the deployment observed when the listing was added. The schema is vendored unchanged in tests/fixtures/mcp_registry/ so validation runs offline.

After review, merge, and a successful MCP deployment, an operator can dispatch Publish MCP registry listing on main. It verifies that the live Worker's serverInfo.version matches the selected commit and writes that version to a temporary manifest. A deployment mismatch fails before authentication or publication. The workflow uses checksum-pinned mcp-publisher v1.8.1 and GitHub OIDC. It has no push trigger and needs no registry secret.

Registry metadata declares a required, secret Authorization: Bearer key. Keys come from pricing and keys.

Sources: registry schema, official publishing workflow, and publisher v1.8.1.

Smithery listing

After review, merge, and a successful MCP deployment, publish the remote URL with the required API-key configuration below. Replace YOUR_SMITHERY_NAMESPACE with the namespace owned by the publishing account:

bash
smithery mcp publish "https://api.modelspec.dev/mcp" \  -n @YOUR_SMITHERY_NAMESPACE/modelspec \  --config-schema '{"type":"object","properties":{"apiKey":{"type":"string","title":"ModelSpec API key","description":"Create one at https://modelspec.dev/pricing/","x-from":{"header":"x-api-key"},"x-to":{"header":"x-api-key"}}},"required":["apiKey"]}'

Smithery collects each user's ModelSpec key and forwards it as the raw x-api-key header value. The MCP Worker converts it to Authorization: Bearer <key> for API calls. This alias applies only to MCP; direct /v1/* API calls still accept only Bearer authentication.

The public server card requires no key. Its serverInfo matches initialize, including the deployed commit as its version. It declares authentication: {"required": true, "schemes": ["bearer"]} and generates its tools from the same registrations as tools/list. The card uses Cache-Control: public, max-age=300.

Sources: Smithery publishing and configuration transport.

SDK and transport

ChoiceWhat
HandlercreateMcpHandler from agents/mcp/server
ServerMcpServer from @modelcontextprotocol/server 2.0
TransportStreamable HTTP (POST /mcp)
StateStateless. No Durable Object.

McpAgent is deprecated and feature-frozen; it exists for sessionful servers that need Durable Object state. These tools are proxies: each call hits the public rank API, policy-check API, or the static JSON export and returns that body plus the origin URL. There is nothing to persist.

Read (2026-09-18):

Route

Workers routes api.modelspec.dev/mcp* and api.modelspec.dev/.well-known/mcp/server-card.json on zone modelspec.dev belong to this Worker. The rank Worker already owns api.modelspec.dev/*; the more specific patterns win. The card route is exact, so discovery clients must use its URL without a query string. Other API routes keep their existing Worker. The CI token can edit Workers routes and cannot create DNS, so there is no new hostname.

Tools

Successful calls pass through the origin. Null on a card means not researched.

ToolOrigin
rankPOST https://api.modelspec.dev/v1/rank
model_infoGET https://modelspec.dev/api/models/<id>.json
list_use_casesGET https://modelspec.dev/api/rank/profiles.json
policy_checkPOST https://api.modelspec.dev/v1/policy-check
decidePOST https://api.modelspec.dev/v1/decide
compare_modelsPOST https://api.modelspec.dev/v1/compare-models
hardware_fitPOST https://api.modelspec.dev/v1/hardware-fit
vocabGET https://modelspec.dev/api/decision/vocabulary.json
feedbackPOST https://api.modelspec.dev/v1/feedback (no key; client is always mcp; the caller's address is forwarded only for the rate limit)

rank, policy_check, decide, compare_models, and hardware_fit use the RANK service binding because a same-zone Worker fetch to the public API hostname reaches the zone origin and returns 522. All five forward the key as Authorization: Bearer <key>. The API validates presented keys and returns its auth/payment errors unchanged.

MCP accepts either Authorization: Bearer <key> or x-api-key: <key>. If both headers are present, their keys must agree; conflicting credentials return HTTP 400 with a JSON-RPC error. Keys never appear in logs or error messages.

All data tools require a ModelSpec key, including when MCP_REQUIRE_API_KEY="false". Missing or malformed credentials return a 401 missing_api_key refusal before any origin call. This includes rank, policy_check, decide, compare_models, hardware_fit, vocab, model_info and list_use_cases. initialize and tools/list remain public. Feedback stays keyless and carries no model data. Use an existing key, or see pricing for availability.

decide accepts the decision spec defined by docs/decision-contract.md. Its MCP input schema comes from the generated docs/decision-contract.schema.json, so the API and MCP contracts cannot drift. Read vocab first for valid facet ids. Use where for Musts that exclude and optimize.weights for Prefers that rank without excluding. Unknown capability values appear in may_qualify. Pin snapshot to reproduce a decision.

compare_models accepts only {models: ["lab/a", "lab/b"]}, with 2 to 4 distinct catalogue ids. It returns the current snapshot's sourced facts, offerings and evidence in caller order. It does not rank or choose a winner. Unknown values stay null, and omitted counts removed records. The entire MCP text stays within 16 KB. An oversized origin response becomes a compact response_too_large tool error and returns no comparison data. This named model comparison does not change the snapshot comparison at /v1/compare.

hardware_fit accepts model, sku, device_count, memory, runtime, quantisation, batch and context. Memory is an optional override in decimal GB per device. Omitted or null inputs reach the API and return undetermined when they prevent assessment. Unknown SKU or quantisation identifiers also reach the API. The tool preserves the endpoint's complete answer and citations. A measured answer cites the exact run. An estimated answer gives estimated_fit: likely|unlikely, a memory margin, the formula, inputs and assumptions. An estimate never proves fit or contains fits: true.

Client config

Run modelspec setup mcp --client claude-code or modelspec setup mcp --client claude-desktop for the client's native snippet. Claude Code:

json
{  "mcpServers": {    "modelspec": {      "type": "http",      "url": "https://api.modelspec.dev/mcp",      "headers": { "Authorization": "Bearer <API_KEY>" }    }  }}

Claude Desktop uses the mcp-remote stdio bridge and requires Node.js/npx:

json
{  "mcpServers": {    "modelspec": {      "command": "npx",      "args": ["-y", "mcp-remote", "https://api.modelspec.dev/mcp",               "--header", "Authorization:${MODELSPEC_AUTH_HEADER}"],      "env": {"MODELSPEC_AUTH_HEADER": "Bearer <MODELSPEC_API_KEY>"}    }  }}

macOS GUI apps do not inherit your shell environment. Explicitly replace the placeholder in the env block with your key, retaining Bearer ; that stores the key in this config. The CLI's setup command writes the placeholder and never copies a saved or environment key into the config. The env block and header form follow the mcp-remote documentation.

Local

bash
npm installnpm testnpm run typechecknpx [email protected] dev --local --var BUILD_COMMIT:dev

Deploy is .github/workflows/mcp.yml on push to main only. Do not deploy from a laptop.

Source: mcp/README.md at commit 7e3d8ba

Tools

9

rank

Legacy v1; use decide. Deprecated fixed-benchmark ranking; scores lag the catalogue and newer models may be unranked. Which models lead a fixed use-case profile? Returns a shortlist with evidence_basis and input provenance, not a quality verdict. Needs an API key (Authorization: Bearer). Costs 1 credit a call; $0.0053–$0.02 per credit by plan or pack (https://modelspec.dev/pricing/). Don't call it as a fallback when decide has no single leader; report its tie. Present `summary_for_user` to the user unchanged and keep every `must_mention` item. When `next_move` is present, offer it instead of a pick. Proxies POST https://api.modelspec.dev/v1/rank with these arguments.

Parameterslimituse_caserequiredconstraintsenvironment

model_info

What does one model's card say? Reads https://modelspec.dev/api/models/<model_id>.json. Split mode returns only its display name. Needs an API key. Not current evidence or a ranking; use decide.

Parametersmodel_idrequired

list_use_cases

Which legacy profiles and ranking rules exist? Returns https://modelspec.dev/api/rank/profiles.json with evidence floors and the neutrality commitment. Needs an API key. Use decide for decisions.

Takes no parameters.

policy_check

Which models and platforms does my policy allow? Returns pass, fail or undetermined per model/platform for licence, origin and data handling; undetermined is not a pass. Needs an API key (Authorization: Bearer). Costs 5 credits a call; $0.0053–$0.02 per credit by plan or pack (https://modelspec.dev/pricing/). It doesn't choose a model; use decide among passing rows. Proxies POST https://api.modelspec.dev/v1/policy-check with these arguments and Authorization.

Parameterslimitmodelsoffsetpolicyrequiredverdictsplatformsrequire_no_undetermined

decide

Which models fit this job? Call decide early. Returns answer, ranked results, may_qualify, reading and recovery hints, decision_id and snapshot. Never add a gate or constraint the user didn't state. Relax only constraints the user stated, one at a time, and say what was relaxed and why. Prefer the answer's own `relax` suggestions. A tie stays a tie. Don't promote a member by knowledge from outside the response. Cite only evidence in the response. Never add benchmark figures from memory. Tied or partial answer: report the tie and caveats, pick no member; a null answer names no pick and says what's missing. A null cost is unknown. Never show it as $0.00 or "free". One vocabulary lookup per unknown facet, then decide. Read `next` before another call. Size `task_tokens` from the user's own task, and say what you assumed. Don't copy the guide's example sizes. Template weights are examples, not requirements. If the evidence basis or board does not match the task, say so: Arena web-dev evidence does not establish chat quality. p_best is the probability of ranking best under the Spec and evidence uncertainty, not the probability of matching the user's task or satisfying its requirements. Read reading when present. Present answer.kind=tied among all answer.members, including with_estate; tie-breakers are conditional. State requirements in reading.not_applied or error.issues that were not applied, including requirements removed after a rejected call. A retry does not verify them. Report estimates as estimates: fits_hardware membership is not measured fit for a specific quantization and context workload. Follow reading.do_not_claim. Never claim a rank the response does not show. Present `summary_for_user` to the user unchanged and keep every `must_mention` item. When `next_move` is present, offer it instead of a pick. Needs an API key (Authorization: Bearer). Costs 1 or 2 credits a call; $0.0053–$0.02 per credit by plan or pack (https://modelspec.dev/pricing/). Not a router: c

Takes no parameters.

compare_models

What facts do 2 to 4 named models have? Returns sourced snapshot facts, offerings and evidence in caller order; unknowns stay null. Needs an API key (Authorization: Bearer). Costs 2 credits a call; $0.0053–$0.02 per credit by plan or pack (https://modelspec.dev/pricing/). It doesn't choose a model and does not rank. Proxies POST https://api.modelspec.dev/v1/compare-models with {models: [...]} at most 16 KB.

Parametersmodelsrequired

hardware_fit

Does this workload fit? Returns measured for an exact cited run, estimated with sources, or undetermined. Uses basis full with sourced headroom, or basis lower_bound for unlikely only. Missing headroom cannot establish likely. Estimates never prove fit. Memory is decimal GB per device, never pooled. Needs an API key (Authorization: Bearer). Costs 2 credits a call; $0.0053–$0.02 per credit by plan or pack (https://modelspec.dev/pricing/). It doesn't choose a model. Proxies POST https://api.modelspec.dev/v1/hardware-fit.

Parametersskubatchmodelmemorycontextruntimedevice_countquantisation

vocab

What can a Spec say? Returns ids and task types. Needs an API key. One vocabulary lookup per unknown facet, then decide. Use decide to compare models. Starter includes a minimal Spec. Compact pages: at most 20 rows, no counts; empty pages end results. Search ranks all tokens before subsets; misses offer nearest ids. Split-mode full details exclude model facts and counts.

Parametersididslimitdetailoffsetsearchsection

feedback

Was an answer reliable? Returns recorded/not_recorded for a rating after acting; include decision_id. Free, no key, and no Authorization header is forwarded. No prompts, keys or personal details in note; no questions or support. Storage stays off until covered by the privacy statement. Proxies POST https://api.modelspec.dev/v1/feedback.

Parametersnoteratingrequiredtemplatedecision_idtrying_to_decide

Version history

1
  1. vb8da9336f3c5a0b56d47987251a29006a5b0487bLatestOct 15, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Model integrations