ModelPick

io.github.Leizhidong-creatorPublished by GitHub account Leizhidong-creatorv0.1.1Updated Nov 19, 2026

Pick the best-value model at every task node, with cost per solved task and the exact setup.

The publisher's own server: model.tools — Zero-Auth Agent Utilities

VerifiedSTDIODesktop onlyDeveloper ToolsAI & ML

Overview

AI-generated overview

Local stdio MCP server that recommends which AI model to use for each task node, with cost per solved task and connection setup.

What you can do
  • Pick and compare models for a task, with quality, cost per solved task and alternatives
  • Plan multi-step workflows with one model per step and explain how a task was classified
  • Browse the model catalog and list the supported task nodes
  • Get connection details for a model: env var, key URL, docs and API model id
  • Return a decision packet for an external judge model, or re-rank a shortlist with a verdict
When to use it
Useful when an agent or developer wants a defensible choice of model per step across text, image, video, 3D and audio, rather than defaulting to one frontier model for everything, and wants the trade-off between quality and cost made explicit.
Requirements
Runs locally over stdio via npx; Node.js 20.11 or newer, with no runtime dependencies. No account, key or network access is needed for recommendations. Provider API keys are only needed to actually run a recommended model, and the server reports which keys are present without reading their values.
Before you install
The catalog is a dated snapshot, not a live price feed, and quality scores come from public benchmarks while expected attempts is a model, so cost figures are estimates. Media quality is the least independently verified part. The setup commands edit agent configuration files, though the README says they back up existing config and preserve sibling servers. It does not proxy requests or store keys.

Installation

In SourceWeft

  1. Open ModelPick in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

[ModelPick — pick the right model at every step]

ModelPick

The model router for AI agents — with cost per solved task.
Coding, image, video, 3D, and audio. One CLI, one MCP server, every agent.

[release] [license] [node] [MCP] [tests] [adversarial] [dependencies]

Website · intro video (93s) · How value is defined · Ecosystem


Your agent can write code, call tools, and ship a feature. Then it picks one model for everything — a frontier model to classify a support ticket, to generate a barrel prop, to draft a thumbnail.

The result is predictable: 60–95% of the spend goes to work that did not need a frontier model, and the hard nodes still fail because the model was chosen for the wrong modality.

ModelPick is a router and cost calculator, not another gateway. Before each node, it answers three questions in one call:

  1. Which model? Ranked for this exact task, with hard capability filters (context, text-in-image, rigging, clip length, open weights).
  2. What will it really cost? Cost per solved task — list price × expected attempts, not $/M tokens.
  3. How do I connect it? The env var, key URL, docs, and API model id.
bash
npx -y github:Leizhidong-creator/modelpick pick "create a stylized 3D sword for a Unity game"

[ModelPick CLI recommending Tripo H3.1 for a 3D game asset]

[Watch the ModelPick intro video]

▶ Watch the intro video with English + 中文 captions · direct MP4 · script & storyboard · English + 中文 captions


Why this exists

Three things are broken in model selection today:

1. Token prices hide the real bill. A model that is 10× cheaper per token but needs 1.8× the attempts and 2× the output tokens is not 10× cheaper. ModelPick computes the denominator that matters: dollars per usable output.

2. A coding agent defaults to one model, even for creative nodes. A 3D pipeline is not a text problem. Text-in-image is not image generation. A 30-second video is not a 5-second clip. ModelPick models 20 task nodes across 5 modalities, with capability filters that reject the wrong model before scoring.

3. Every tool is a new integration. ModelPick ships as a zero-dependency CLI, a stdio MCP server, a portable SKILL.md, and a Claude Code plugin. Same decision engine everywhere; no gateway, no lock-in, no key storage.

In one line: other tools route text prompts. ModelPick routes work — coding, image, video, 3D, and audio — and prices each node per solved task.

How is this different from a gateway or a coding router?

Gateway (OpenRouter, LiteLLM, Portkey)Coding router (claude-code-router)ModelPick
Optimizesaccess, failover, provider marginmodel switching inside a coding loopbest price-performance at each node
Modalitiesmostly text; OpenRouter's live catalog is ~4% non-texttexttext, image, video, 3D, audio
Decisionroute by price/availabilityrules or a classifiercapability filters + benchmark-weighted utility
Keysheld by the gatewaylocalnever stored; you call the provider
Evidencelive pricesconfigdated catalog + public benchmark + raw failures

Gateways are useful, and ModelPick can recommend one as the setup path. But a gateway cannot tell you that a 3D node should use Tripo instead of Meshy, or that a "headline text" image needs a model with a text-rendering score above 90. That choice is what this tool makes.

More: research/competitive-landscape.md (15 products, 10 features, 53 sources).


30-second quickstart

bash
# 1. Ask for a recommendation (no key, no signup, no network)npx -y modelpick pick "generate a product photo with the headline 'Fresh Roast'"
# 2. Plan a multi-step deliverablenpx -y modelpick plan "30 second product launch video"
# 3. Install into your agent (writes MCP config + skill)npx -y modelpick setup claude-code

Official npm package: https://www.npmjs.com/package/modelpick

MCP Registry: https://registry.modelcontextprotocol.io/v0/servers?search=modelpick

Fallbacks if the registry is unavailable:

bash
# GitHub, no npm authnpx -y github:Leizhidong-creator/modelpick pick "refactor this 40k-line repo, cheapest that works"
# static registry mirrornpm install modelpick --registry=https://leizhidong-creator.github.io/modelpick/npm/
# offline release tarball
```bashnpm install https://github.com/Leizhidong-creator/modelpick/releases/download/v0.1.3/modelpick-0.1.3.tgz

[ModelPick planning a product launch video with a model per step]


Works with the agents you already use

modelpick setup writes the files for the agent you use. It backs up existing config, keeps sibling MCP servers, and refuses to overwrite invalid JSON.

AgentCommandMCP file it uses
Claude Codesetup claude-code.mcp.json (project) or ~/.claude.json (--global)
OpenAI Codex CLIsetup codex + codex mcp add~/.codex/config.toml
Cursorsetup cursor.cursor/mcp.json
DeepSeek Harnesssetup dsh --global --write --profile desktop~/.dsh/profiles/<profile>/cordis.patch.yml
GitHub Copilot CLIsetup copilot.mcp.json
VS Code / Copilot agent modesetup vscode.vscode/mcp.json (servers)
Any MCP clientmodelpick mcpyour client's stdio config

Claude Code

bash
npx -y github:Leizhidong-creator/modelpick setup claude-code# writes .mcp.json, .claude/skills/modelpick/, and a CLAUDE.md blockclaude mcp list        # modelpick should appear

Restart Claude Code, run /mcp, and approve the project server. Then ask: pick a model for a 3D sword for Unity.

Plugin alternative, inside Claude Code:

text
/plugin marketplace add Leizhidong-creator/modelpick/plugin install modelpick@modelpick

OpenAI Codex CLI

bash
npx -y github:Leizhidong-creator/modelpick setup codex# or write the global MCP row directly, with a backup:npx -y github:Leizhidong-creator/modelpick setup codex --global --writecodex mcp list

setup codex writes AGENTS.md and .agents/skills/modelpick/. Codex reads MCP servers from ~/.codex/config.toml, so --global --write appends the row or you can use codex mcp add:

bash
codex mcp add modelpick -- npx -y github:Leizhidong-creator/modelpick mcp

Cursor

bash
npx -y github:Leizhidong-creator/modelpick setup cursor

It writes .cursor/mcp.json, .cursor/rules/modelpick.mdc, .cursor/skills/modelpick/, and AGENTS.md. Open Cursor → Settings → MCP and confirm modelpick is connected.

DeepSeek Harness

DSH has no .mcp.json. MCP servers are Cordis rows in the profile patch:

bash
npx -y modelpick setup dsh --global --write --profile desktop

It writes the portable skill to ~/.dsh/skills/modelpick/, appends AGENTS.md, and adds one MCP row to ~/.dsh/profiles/desktop/cordis.patch.yml (backing the file up first).

Restart DSH after this. Then ask it:

text
Use modelpick pick_model for a low-poly sword for Unity. Show costPerTask and the key.

Without --write, setup dsh prints the row instead of editing a live profile. The MCP server itself can be tested without DSH:

bash
node bin/modelpick.js mcp# then send initialize + tools/list over stdio

GitHub Copilot CLI

bash
npx -y github:Leizhidong-creator/modelpick setup copilotcopilot mcp list

It writes .mcp.json, .github/skills/modelpick/, and AGENTS.md. Restart Copilot and ask for a model pick.

VS Code / Copilot agent mode

bash
npx -y github:Leizhidong-creator/modelpick setup vscode

It writes .vscode/mcp.json under servers (VS Code's schema, not mcpServers), the portable skill, and AGENTS.md. In VS Code run MCP: List Servers, start modelpick, then ask in agent mode. Sibling servers are preserved.

Any MCP client

json
{  "mcpServers": {    "modelpick": {      "command": "npx",      "args": ["-y", "github:Leizhidong-creator/modelpick", "mcp"]    }  }}

Tools exposed: pick_model, get_decision_packet, plan_workflow, compare_models, get_setup, list_models, explain_task. Recommendations work without any provider key; the key is only needed to run the chosen model.

Portable skill: .agents/skills/modelpick/SKILL.md is read by Codex, Cursor, Copilot CLI, Devin/Windsurf, and DeepSeek Harness. Claude Code gets its own copy. The skill tells the agent when to call pick_model and how to report the result.


The number that matters: cost per solved task

raw cost per attempt = tokens × price (cache-aware)   OR   units × unit priceexpected attempts    = 1 / (1 − 1.15 × (100 − quality) / 100)     capped at 4cost per solved task = raw cost × expected attempts

Worked example — 80K input / 8K output, 60% cache hit, agentic coding:

Modelqualityraw costexpected attemptscost per solved task
Claude Fable 5.292.3$0.591.10$0.64
Gemini 4 Carbon87.3$0.151.17$0.17
DeepSeek V4.1 Flash70.7$0.0101.51$0.015
Claude Haiku 5.568.8$0.0081.56$0.012

The cheap model is 53× cheaper and 23.5 quality points worse. The interesting answer is the third row: mid-quality agentic coding at 2.3% of the frontier cost. The right answer depends on the node — which is why the tool shows both sides instead of hiding the trade-off behind a token price.

Quality is a weighted mix of public benchmark signals (BenchLM overall/coding, CursorBench 4.0, capability-specific scores). Expected attempts is a transparent model, shown on every result — not a promise. If a number is estimated, the CLI prints ~.


What it covers

40 models · 18 providers · 20 task nodes · 5 modalities. Current as of 2026-11-19; every price carries a source and read date.

ModalityNodesExample models in the catalog
Textcoding, review, refactor, debug, plan, long-context, research, writing, translation, extraction, classification, data, computer-useClaude Fable 5.2, GPT-6.2 Astra/Sol, Gemini 4 Carbon/Hydrogen, DeepSeek V4.1 Flash, Claude Haiku 5.5, Kimi K3.1 Pro
Imagegeneration, editing, text-in-imageNano Banana 2.1/Pro, GPT Image 2.6, Seedream 5.5
Videotext/image → clip, dialogue, 30s takesVeo 3.1 / Fast (Vertex), Kling 4.0, Seedance 2.5
3Dtext/image → game-ready asset, riggingTripo H3.1, Hunyuan3D 3.1 Pro, Meshy 7.1, TRELLIS 2 (open)
AudioTTS, STT, musicEleven v4, Voxtral TTS 3, GPT Transcribe, Voxtral Transcribe 3, Lyria 3.5, MiniMax Music 3

The catalog is a dated snapshot, not a live price feed. modelpick update refreshes from the repo; modelpick doctor warns when it is stale.


Tested decisions

[ModelPick decision benchmark: 115 runs, 100% top-1 acceptable, 100% hard constraints]

MetricDevelopment setHeld-out setTarget
Top-1 acceptable100% (115 runs)100% (35 runs)≥75%
Top-3 acceptable100%100%≥92%
Intent accuracy100% (39 scenarios)100% (12 scenarios)≥95%
Hard constraint pass100%100%100%
Price stability (±20%)100%100%≥80%
p95 decision latency1.1 ms1.1 ms≤300 ms

What this is: 39 expert-labeled scenarios expanded into balanced/cost/quality runs, plus a 12-scenario held-out set authored after the benchmark froze. Every run checks task detection, top-1/top-3 acceptability, hard constraints, cost regret, and stability under ±20% price perturbation.

What this is not: a live model-output benchmark. It does not call providers, and it does not prove quality labels are universally correct. Quality scores are derived from public leaderboards; expected attempts is a model. The full methodology, failures, and reproduction commands live in tests/TEST_PLAN.md, tests/results/, and bench/.

[Quality versus cost per solved task for coding models]

Full per-node tables: docs/LEADERBOARD.md.

Value, measured

node bench/value.js simulates 4,000 jobs per task across five task types, using catalog quality as first-pass success and the router's retry curve:

StrategyFirst-passCost / solvedQuality parityCost vs frontier
Frontier always93.7%$0.6417100%100%
Value pick89.4%$0.256395.4%41.6%
Cheapest always87.6%$0.206093.5%39.1%

The value pick keeps 95.4% of frontier first-pass quality at 41.6% of the cost. The cheapest model saves 2.5 cost points but loses 1.9 quality points and fails a 90%-parity bar on two of five tasks. This is synthetic; the arithmetic and the definition are in docs/VALUE.md, and the evidence behind every quality score is in docs/QUALITY.md.

Multi-dimensional scorecard

tests/results/SCORECARD.md combines six dimensions from committed evidence:

CorrectnessCostRobustnessCoverageSafetyOperabilityComposite
10084.210010010010096.8 / A

[ModelPick six-dimension scorecard]

Correctness is the decision benchmark; it is not an independent output benchmark. Cost is regret plus the value simulation. Robustness is price perturbation, retry sensitivity, and determinism. Coverage is every task and model. Safety is the 25-case adversarial suite. Operability is p95 latency, modelpick verify, and line coverage (84.8%).


The decision engine

request  │  ├─ task detection (20 nodes, English + Chinese) ── explicit --task always wins  ├─ intent parsing (cheap / fast / best / open-source / budget / exclude)  ├─ hard requirements (context, text-in-image, rigging, lip-sync, clip length)  │  ├─ capability filter → candidates  ├─ benchmark-weighted quality  ├─ cache-aware cost × expected attempts  ├─ latency + context fit  └─ weighted utility → pick + alternatives + why + trade-offs + setup

Presets: balanced (task weights), value (price-performance), quality, cost, speed, privacy. Budgets and quality floors are hard constraints, not suggestions.

The default is a value decision, not a bargain hunt. The value pick is the cheapest model within 8 quality points of the best eligible model; if the band is empty, it falls back to the top recommendation. It is returned as picks.value and value, with the quality gap and the percentage of the best model's cost. The full definition, simulation, and sensitivity analysis are in docs/VALUE.md.

Every answer includes:

  • recommendation.slug, quality, costPerTask, expectedAttempts, latencySec
  • why[] and tradeoffs[]
  • setup.direct (env, key URL, docs, API model) and an OpenRouter fallback for text models
  • catalog.version, stale, priceConfidence

Bring your own decision model

The deterministic router is the default. If you want a judge or decision model (Claude, GPT, Gemini, a Google decision model, or a local model) to re-rank the shortlist, keep it outside the key store:

bash
modelpick decide "rigged 3D character for Unity" --json > packet.json# send packet.prompt to your judge; save its JSON as verdict.jsonmodelpick pick "rigged 3D character for Unity" --verdict verdict.json
# Decision-model candidates with exact endpoints and prices:modelpick judges

judges includes Jev (System One) (jev-latest, Venice, $0.042/1M input) and Gemini Decision 1 (gemini-decision-1, $0.10/1M input, zero output). They are decision models, not generation models.

The verdict can only reorder the shortlist, at most two positions, and cannot invent a model or override a hard constraint. pick_model accepts the same verdict object over MCP. The protocol, guardrails, and meta-evaluation thresholds are in docs/JUDGES.md.


Commands

text
modelpick pick "<task>"            Rank models for one nodemodelpick plan "<goal>"            Multi-step workflow, one model per stepmodelpick compare <a> <b> --task   Side-by-side cost per solved taskmodelpick models                   Browse the catalogmodelpick tasks                    List the 20 nodesmodelpick show <model>             Endpoint, auth, SDK, curl, and evidencemodelpick decide "<task>"          Build a portable decision packet for any judgemodelpick verify                   Verify catalog, evidence, benchmarks, hashesmodelpick eval <file>              Analyze your own pass/fail runsmodelpick add-model <file>         Add a private/custom modelmodelpick setup [agent]            Install MCP + skillmodelpick mcp                      Run the stdio MCP servermodelpick keys                     Show required env vars (never values)modelpick doctor                   Diagnose install, keys, freshnessmodelpick update                   Refresh the catalog snapshotmodelpick bench                    Run the decision benchmark

Useful flags: --optimize, --max-cost, --min-quality, --open-weights, --context, --exclude, --task, --verdict, --json, --explain, --no-color.

bash
# Machine-readable for any agentmodelpick pick "edit this product photo, remove the background" --json
# Quality floor + budgetmodelpick pick "implement OAuth" --min-quality 85 --max-cost 0.25
# Open weights onlymodelpick pick "refactor this repo" --open-weights
# Only models whose key is already in this shellmodelpick pick "refactor this repo" --available
# Need 400K tokens of contextmodelpick pick "read this 400-page document" --context 400000

Only have a few API keys?

You do not need every provider. ModelPick separates the best decision from what you can run right now:

bash
modelpick pick "refactor this repo"
text
! key missing: GEMINI_API_KEY, GOOGLE_API_KEY — recommendation is still validrun now  GPT-6.2 Sol · $0.20 · quality 83.9 · key OPENAI_API_KEY
  • --available restricts ranking to models whose key exists (or any text model if you have OPENROUTER_API_KEY).
  • The JSON always includes availability.bestAvailable, availability.fallbacks, and recommendation.availability.missing.
  • An OPENROUTER_API_KEY is a universal fallback for text models; FAL_KEY covers many media models.
  • If a capability has no accessible model, the tool says so instead of inventing a recommendation.

Add your own model

Bring a private endpoint, a self-hosted vLLM/Ollama model, or a provider the catalog does not cover:

bash
modelpick add-model --example > my-model.json# edit the slug, provider env var, pricing, and quality scoresmodelpick add-model my-model.jsonmodelpick pick "refactor this repo"          # it is now a candidatemodelpick add-model --remove my-model

The model is written to ~/.modelpick/overrides.json, validated against the full catalog before saving, and never touches the repo. You can also add a whole list with { "models": [...] }.

Connect without guessing the endpoint

pick gives you the env var and key URL. show gives you the full connection card:

bash
modelpick show tripo-h3.1
text
CONNECTendpoint  https://api.tripo3d.ai/v2/openapimodel id  h3.1auth      bearer · env TRIPO_API_KEYplatform  https://platform.tripo3d.aidocs      https://developers.tripo3d.ai/en/docssdk       REST / MCP
examplecurl -X POST "https://api.tripo3d.ai/v2/openapi/task" \  -H "Authorization: Bearer $TRIPO_API_KEY" \  -H "content-type: application/json" \  -d '{"type":"text_to_model","prompt":"a stylized sword"}'

The same object is returned by the MCP get_setup tool. If an endpoint is a provider's common path rather than a freshly verified one, the card says so (typical vs verified).


Built to change

Model prices move weekly. New providers appear monthly. The engine is deliberately data-first:

LayerFileChange without code
Models, prices, IDscatalog/models.jsoncatalog/quality.json + scripts/sync-catalog.js
Task nodes, weights, floors, anchorscatalog/tasks.jsonedit JSON, run npm run validate
Retry curve, presets, anchorscatalog/policy.jsonedit JSON
Your private models / price overrides~/.modelpick/overrides.jsonadd a full model object or patch a slug
Workflowscatalog/plans.jsonadd steps referencing task ids
Agent adapterssrc/integrations/one function per agent

Custom model example:

json
{  "models": [    {      "slug": "my-private-model",      "name": "My Private Model",      "provider": { "id": "internal", "name": "Internal", "env": ["INTERNAL_KEY"], "keysUrl": "https://…", "docsUrl": "https://…" },      "modality": "text",      "contextWindow": 200000,      "pricing": { "unit": "mtok", "input": 1, "output": 2, "currency": "USD" },      "quality": { "overall": 88, "coding": 88, "codingAgent": 88, "reasoning": 88, "longContext": 80, "writing": 80, "multimodal": 60, "cheapWork": 70 },      "speed": { "outputTps": 50, "ttftMs": 1000 },      "tags": ["benchmarked"],      "openWeights": false,      "sources": [{ "name": "internal", "url": "https://…", "readAt": "2026-11-19" }]    }  ],  "exclude": ["model-you-never-want"],  "policy": { "presets": { "cost": { "quality": 0.35, "cost": 0.65, "speed": 0, "context": 0 } } }}

No fork. No rebuild. modelpick doctor validates the result.


Data, honesty, and limits

  • Prices: models.dev / OpenRouter for text; BenchLM media pricing, Google Vertex, OpenAI, fal, Tripo, Meshy, ElevenLabs for media. Read date on every source.
  • Quality: BenchLM BenchAlign and CursorBench 4.0 for text; vendor specs plus public comparisons for media. Media quality is the least independently verified part of the catalog; it is labeled and dated.
  • Known omissions: Veo is Vertex-only now; Sora 2 is discontinued; GPT-Image-1 and Imagen 4 are retired. These are deliberately not in the catalog.
  • No telemetry. No key storage. Recommendations work offline. keys reads presence only.
  • Not a gateway. ModelPick does not proxy requests, mark up tokens, or hold your data. You call the provider directly.

See docs/DATA.md for provenance and research/ for the underlying research.


Development

bash
node scripts/validate.js       # catalog + policy + manifestsnode --test "tests/**/*.test.js"node bench/run.js              # decision benchmark -> tests/results/node bench/holdout.js          # held-out decision benchmarknode bench/value.js            # cost-per-solved-task simulationnode tests/adversarial/run.js  # 28 adversarial casesnode scripts/live_smoke.js      # free/public endpoint smoke (network)node scripts/check.js          # all of the abovenode scripts/make_assets.py    # regenerate README/test images

Requirements: Node ≥ 20.11. Runtime dependencies: zero. Python + Pillow are only needed to regenerate images.


Roadmap

  • Publish to npm (npx modelpick) and submit to MCP directories
  • Live price refresh from provider APIs with signed snapshots
  • Community-contributed task nodes and quality evidence
  • Router mode: optionally execute the recommended call through the user's own key
  • Cost ledger: track actual spend per node across a project
  • Windows-native MCP shim (no cmd /c)

If ModelPick saves you money, star the repo — it is the only distribution channel for a tool with no VC and no telemetry.

MIT © 2026 Lei Zhidong · built with zero runtime dependencies and a healthy distrust of token pricing.

Source: README.md at commit 6e5821d

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.1.1LatestNov 19, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Model integrations