Legion

Legion

"I am Legion, for we are many."

An MCP-native model council. Legion exposes LLMs as individual tools and orchestrates them into debates, juries, blind panels, private refinement gauntlets, workshops, and custom multi-model deliberations.

Every model is reached through the OpenAI Responses API wire format. Use OpenAI or Azure directly, route other providers through a compatible gateway (such as a LiteLLM proxy), and configure the entire council through hot-reloadable files.

Contents

• How it works
• Design decisions
• Requirements
• Setup
• Configuration
• Logging
• Run
• Try it
• Use in VS Code
• Deploy

How it works

flowchart LR   AI[Calling AI] -->|claude / gpt / gemini …| Legion   Legion -->|Responses API| GPT[OpenAI / Azure — direct]   Legion -->|Responses API| GW[Gateway e.g. LiteLLM]   GW --> Claude & Gemini & Llama

• One tool per model, named after the slugified model name (e.g. Claude → claude). Each accepts a prompt plus optional context, role, system, temperature, and maxTokens.
• A quorum tool fans one prompt out to several models — with roles, multi-round discussion, visibility modes, and synthesis — and returns each answer separately. See Presets for the orchestration options.
• Presets are named, pre-staffed councils (debate, courtroom, code review, …), each exposed as its own tool.
• Identity and telemetry ride in structuredContent, not the answer text. Logging goes to stderr (safe for stdio).

Design decisions

• No provider adapters. There is no provider-specific code and no built-in model list. Legion speaks one wire format; models that don't speak it natively go through a gateway. Supporting a new model requires no change here.
• Models are config, not code. Adding a model means adding a JSON file. Over HTTP the directory is re-read per request, so no rebuild or restart; over stdio the config is read once per connection (see Hot-reload).
• One tool per model. Each model appears to the calling AI as its own tool with its own description, rather than a single tool with a model parameter. The quorum tool covers the ad-hoc multi-model case, and each preset in config/presets/ is exposed as its own enforced, pre-staffed council tool.
• Stateless. Every call is one-shot with store: false. Nothing is persisted, so there is no database and no conversation state to manage.
• Small. A couple thousand lines of TypeScript, one bundled output file, five dependencies.

Requirements

• Node.js 24+
• At least one OpenAI-Responses-compatible endpoint (a provider API directly, or a gateway such as LiteLLM for models that need bridging)

Setup

From npm — no clone, no build:

npx legion-mcp

From a clone:

npm installcopy .env.example .env   # then edit .env

Configuration

All configuration lives in a config/ directory. The bundled defaults are always the base layer; a config/ folder in the current working directory is overlaid on top of them, per file:

• Directory resources (models/, roles/, presets/, tools/): a local file overrides the bundled file of the same name; a local-only file is added; every bundled file you don't touch stays. So dropping in one config/presets/refine.json overrides just that preset — the other bundled presets remain.
• Single-file text (prompts.json, errors.json, schema.json): merged per key — bundled < local. A partial local file overrides only the keys it sets.
• description.md: local wins whole if present, else bundled.

The overlay can override or add, but not delete a bundled entry. To choose which bundled presets register as tools, use PRESETS (see below).

Installing from npm? You must supply your own model files. The bundled config ships only key-free *.example.json model files, which the scanner deliberately ignores — so the bundle contributes zero real models. With no real model file the server fails fast at startup (No model files found in ...). Drop one config/models/<name>.json next to where you run the server (see below) — the rest falls back to the bundled defaults.

The layout below is identical either way.

Hot-reload

Config is read fresh when a server instance is built, never baked in at build time — but when that happens depends on the transport:

HTTP builds a fresh server per request, so a dropped-in model, role, preset, or text override is picked up by the very next call. Stdio pins one server for the life of the connection, so the same edit lands when the client restarts Legion — which for most desktop MCP clients means reloading the server, not rebooting the machine.

Where this README says something is "live on the next call", read it as the HTTP behavior; over stdio it's the next connection.

Models — config/models/*.json

At least one model file is required — the server fails fast without one. Each JSON file becomes a tool, named after the slugified file name (config/models/fable.json → tool fable):

{   "model": "claude-fable-5",   "description": "Claude Fable — fast, creative, general purpose.",   "baseUrl": "https://api.example.com",   "apiKey": "sk-optional-per-model-key"}

• model (required) — the deployed model id the endpoint routes to.
• description — helps the calling AI pick the right model.
• system — optional baseline system instructions baked into every call to this model.
• baseUrl / apiKey — optional; omitted values fall back to DEFAULT_BASE_URL / DEFAULT_API_KEY. apiKey may reference an environment variable with the env: prefix — "apiKey": "env:GPT_KEY" reads GPT_KEY from the environment (unset/empty is a fatal startup error), so a key can stay out of the file entirely.
• omitParams — optional list of request params to drop for this model, e.g. ["temperature"]. The server stays provider-agnostic: it never assumes which models reject which params — you declare each model's quirks here. Useful for reasoning models and some deployments that reject temperature.
• reasoning — optional "minimal" | "low" | "medium" | "high", sent as reasoning.effort. Reasoning models can spend an entire maxTokens thinking and emit nothing; "low" may help leave room for an answer. The endpoint must support this parameter; it does not bypass content filtering.
• maxTokens — optional hard output ceiling for this model. Every request sends min(requested, ceiling), whatever the caller or preset role asked for. For models that think harder the more room they get and come back empty.

Hot-drop: the directory is re-scanned per request — add or edit a model file and it's live on the next call, no restart.

Secrets & git: model files can contain API keys, so config/models/*.json is git-ignored. Copy a *.example.json (tracked, key-free, ignored by the scanner) to get started:

copy config\models\gpt.example.json config\models\gpt.json   # then add your key

Roles — config/roles/*.md

Optional hot-droppable instruction files. Each .md file becomes a named role (slugified from filename). Drop a file, it's live on the next call. This repo ships skeptic.md, builder.md, judge.md, and short.md (a terse "answer immediately, no deliberation" role useful for constrained-output turns) as ready-to-use starters — edit or delete them freely (they hold no secrets).

Available selectors in tools become roleName, e.g. passing role: "skeptic" or using "model:skeptic" in quorum.models.

Presets — config/presets/*.json

Optional hot-droppable council recipes, one JSON file per preset (named after the slugified file name, like models). Each preset becomes its own tool — drop config/presets/code_review.json and a code_review tool appears on the next request. Each preset has a description, a roles list, and optional authoritative mode / synthesizer defaults. Each role defines its behavior inline — a role's description is its instructions (the behavior contract); a role with no description falls back to a matching config/roles/<role>.md file:

{   "description": [      "Free-for-all: pit several contestants against each other, then crown a winner.",      "",      "Staff `contestant` with as many models as you like; one `judge` decides."   ],   "mode": "parallel",   "synthesizer": "judge",   "roles": [      { "role": "contestant", "description": "Argue why your answer beats the others.", "min": 2, "max": null },      { "role": "judge",      "description": "Crown a single winner and justify it.", "min": 1, "max": 1 }   ]}

The calling AI invokes the preset tool directly (e.g. code_review) and still writes the models selectors, assigning any model to any preset role. Presets are enforced: every selector must use a preset role and every role must be staffed within its cardinality, else the result is an error saying what to fix.

Keys:

Role object keys:

For example, a courtroom call assigns lawyers to sides with @team tags. The first lawyer listed for each side gives that side's closing statement:

{   "models": [      "gpt:lawyer@prosecution",      "grok:lawyer@prosecution",      "claude:lawyer@defense",      "kimi:juror",      "llama:juror",      "mistral:juror",      "opus:judge"   ],   "objectives": {      "prosecution": "Prove liability.",      "defense": "Defeat liability."   }}

Authoring notes

Things that bite when writing a preset:

• Eliminations use exact labels. The judge copies one CUT <label> command from the eligible menu, then gives a reason on the next line. Optional cuts also allow KEEP ALL. Bare numbers and ambiguous selections are rejected. Local elimination prompt overrides must use this command format.
• Required elimination can stall. Bounded cleanup retries do not guarantee a sole survivor. An unresolved run retains its summary and discussion, but returns isError: true and structuredContent.incomplete with the remaining labels. Optional-elimination contests are not subject to this requirement.
• min/max count every speaker in that role, not per team. They bound the whole role across all teams, so a tag-team wrestler role that must cover sides from a 2v2 up to a 6v6 is min: 4, max: 12 — a 3v3 is just one valid staffing (6 wrestlers) inside that range, not its own max: 3. The engine cannot enforce "even sides" or "one per side" — say it in the description instead.
• Neutral roles cannot be teamed. The eliminator, framer, and a single-seat synthesizer reject a @team tag, so a role that belongs to one side can't hold those jobs.
• Some keys require others, and a violation is caught at load: synthesizeEvery needs a synthesizer, eliminateEvery needs an eliminator, reframeEvery needs a framer, closingLast needs closing, vote options need vote, and a single-seat synthesizer needs at least one other required role (it sits out the normal rounds). enterEvery and tagTeam cannot be combined — both decide who speaks.
• A malformed preset is skipped, not fatal. It logs ❌ preset skipped — Invalid <file>: <reason> and every other tool still registers, so check the server log when a preset tool doesn't appear.
• Cost is speakers × rounds serial model calls, so defaultRounds and a generous max multiply quickly. mode: "parallel" collapses each round to its slowest speaker, at the price of speakers no longer seeing each other within a round.

This repo ships these presets — edit or delete freely:

battle_royale

Free-for-all contest; an overseer crowns a winner.

brainstorm

Divergent idea generation across models.

bullying

One model defends a position while the rest gang up on it; an optional teacher rules on whether it held.

code_review

Structured multi-model code review.

courtroom

Team-tagged lawyers argue opposing sides, jurors vote by side, and a judge rules.

debate

Opposing sides argue a question to a synthesis.

double_blind

Independent blind panel — no one sees the others.

election

Candidates campaign, then the field decides by secret ballot — the anonymous vote is the verdict, not a judge's call. Optional incumbent defends a record; an optional silent electorate reads every round and votes without campaigning.

final_girl

Survivors culled one per round until one remains.

focus_group

Moderated panel that riffs off each other.

gauntlet

Private self-refinement race across rounds.

quick_take

Fast two-voice gut check — one direct answer, one line of pushback.

refine

Relay polish of an existing artifact.

tag_team

Tag team match: one @team-tagged wrestler is legal per team each round and the rest wait on the apron, rotating every round so a three-person side cycles through all three. An optional run_in hits the ring for a single round (book it with cameoRound), an optional announcer calls the match, and the ref names the winning team.

tiebreak

A decisive third voice resolves a stalemate.

war_games

A staggered-entry team cage match: @team-tagged combatants enter one at a time while a neutral ref calls fouls and names the winning team, with an optional booker who sets the match.

workshop

Differentiated creative team.

Which bundled presets register as tools is controlled by PRESETS — unset registers them all, and a local-only preset (a slug with no bundled counterpart) you add under your own config/presets/ is always registered regardless, since authoring one is the opt-in. A local file that reuses a bundled slug is a customization of that preset, not a new one, so it still obeys PRESETS. With no bundled presets and no local presets there are no preset tools — but note the bundled set is the base layer, so you get every bundled preset even when you ship no config/presets/ folder of your own.

Role text nudges output, it doesn't cap it — use maxTokens for a hard per-turn limit, and budget generously for reasoning models and multi-round quorums. tokenBudget is a different axis: it counts every call's input and output for the whole run, so size it from turns × (context + maxTokens) and set it above the estimate — input dominates once the transcript grows.

AI guidance — config/description.md

Optional markdown served to clients as MCP instructions — describe your models and when the AI should use each. See this repo's copy for a template. The literal {longRuns} marker is replaced with config/sync/long-runs.md or config/async/long-runs.md depending on ASYNC_TOOLS, so the delivery guidance (block-and-wait vs start/poll/cancel) tracks the mode without duplicating the rest of the file. A local description.md without the marker is served as written; a startup warning notes that mode guidance is missing.

Tool, field & message text — config/*.json and config/tools/*.md

The text a caller reads to drive Legion — tool and field descriptions, prompt scaffolding, runtime errors — lives in config, not code. (Result-shape descriptions in outputSchema, generated staffing lines, and startup errors stay in code.) Each file merges over the bundled JSON base per key, so override only what you want; open the shipped copies to see the full key set and {token} placeholders:

• config/tools/quorum.md, poll.md, cancel.md — descriptions for the quorum tool and (async mode only) the poll / cancel tools. Delete one to fall back to its built-in string. Model tools describe themselves from their model file's description, and preset tools from the preset's own.
• config/sync/long-runs.md, config/async/long-runs.md — the mode-specific block spliced into description.md at {longRuns}.
• config/schema.json — input-field descriptions (prompt = shared fields, quorum = quorum-only; a quorum key wins on a name clash).
• config/prompts.json — the prompt-shaping templates models read: role contract, context block, transcript header, round banners. Tune how strongly roles bind and how rounds are framed here.
• config/errors.json — runtime error messages shown to the calling AI.

(Startup/config-validation errors stay in code — a message that reports a broken config file can't live inside it.)

Environment variables

* Every model must resolve a baseUrl and apiKey from its file or the defaults — validated at startup.

The server fails fast at startup on a missing/empty models directory (unless ALLOW_NO_MODELS=true), invalid model files, an unresolvable endpoint or key, or two file names that slugify to the same tool.

Routing

Every tool call is a stateless, one-shot Responses API request. Models whose endpoints natively speak Responses (OpenAI, Azure OpenAI / Foundry) set a baseUrl to be called directly; the rest fall back to the defaults — typically an OpenAI-compatible gateway like LiteLLM that bridges to their native APIs.

Logging

• info (blue): server start and one metadata line per model call — model, latency, token usage, role, context presence. No prompt/response content.
• debug (gray): additionally logs the full prompt and response (context is noted as present, not printed).
• warn (orange) / error (red): fallbacks and failures.

Color is auto-disabled when stderr is not a TTY.

Run

One entrypoint; the transport comes from MCP_TRANSPORT (http is the default, set stdio for desktop MCP clients).

From npm (legion-mcp bin — run from a directory holding your config/):

npx legion-mcp                             # Streamable HTTP transport on :$PORT/mcp$env:MCP_TRANSPORT='stdio'; npx legion-mcp # stdio transport

Installed globally or as a dependency, the same binary is on PATH:

npm install -g legion-mcplegion-mcp

Development (no build step, via tsx):

npm run dev         # Streamable HTTP transport on :$PORT/mcpnpm run dev:stdio   # stdio transport

Production (compiled to bin/server.js):

npm run buildnpm start           # httpnpm run start:stdio # stdio

Async mode — ASYNC_TOOLS=true

A council can run for minutes; hosted ChatGPT and Codex abandon a tool call long before that (observed ~124 s; Codex defaults to 60 s). With ASYNC_TOOLS=true the quorum and preset tools validate the request, start the council in the background, and return at once with a handle — structuredContent.jobId, the current state/phase, and a suggested pollIntervalMs. Two extra tools appear:

• poll — read-only; returns the job's state, the answers and public notes completed so far, and (once terminal) result: the full council output in its usual shape. full: true adds the rendered transcript and timeline.
• cancel — aborts in-flight model calls, returns whatever had completed, and settles the job as cancelled with no synthesis. Idempotent.

Individual model tools stay synchronous. Server instructions switch to the start/poll/cancel workflow so the calling model knows what to do. Jobs are held in process memory under JOB_* bounds — a restart loses them, and a request routed to a different replica cannot see them, so run a single instance (or pin sticky routing) when this mode is on. Set TRUST_PROXY_AUTH=true behind Azure Container Apps Easy Auth to bind each job to the authenticated caller; without it, possession of the jobId is the only access control.

Try it

List the tools with the MCP Inspector:

npx @modelcontextprotocol/inspector -e MCP_TRANSPORT=stdio npx tsx ts/server.ts

Use in VS Code

Add to your mcp.json — from npm:

{   "servers": {      "legion": {         "command": "npx",         "args": ["-y", "legion-mcp"],         "cwd": "path/to/your/config/parent",         "env": {            "MCP_TRANSPORT": "stdio",            "DEFAULT_BASE_URL": "https://your-gateway.example.com",            "DEFAULT_API_KEY": "sk-your-key"         }      }   }}

Or from a clone:

{   "servers": {      "legion": {         "command": "node",         "args": ["bin/server.js"],         "cwd": "path/to/legion",         "env": {            "MCP_TRANSPORT": "stdio",            "DEFAULT_BASE_URL": "https://your-gateway.example.com",            "DEFAULT_API_KEY": "sk-your-key"         }      }   }}

For the HTTP transport, point your client at http://<host>:<PORT>/mcp.

Health

• GET /health — cheap liveness: confirms the process is up and config loaded. Returns { status: "ok", name, version, models } (a count). Makes no external calls. This is what container HEALTHCHECKs and Kubernetes liveness/readiness probes should hit.
• GET /health?deep — optional connectivity check: sends a tiny prompt to every model and reports per-model reachability (503 if any fail). Makes a real billable call per model, so use it manually — don't wire it to an automatic probe.

Deploy

Ready-to-use container deployment examples (Azure App Service, Azure Container Apps, Docker Compose, Kubernetes, and Compose + Caddy for HTTPS) live in examples/ — each installs Legion from npm and ships a complete drop-in config/.