DOS — the trust substrate for agent fleets

DOS — the Dispatch Operating System

Catch your AI agents when they lie about what they shipped.

[PyPI] [Python versions] [CI] [verified by DOS] [commit-claims] [License: MIT]

📊 See it run on real repos: the scoreboard scores 15 popular AI-built repos (roborev, open-interpreter, crewAI, autogen, …) — how much agents wrote, which ones, and whether each commit's claim is backed by its own diff. Score yours: dos commit-audit --sweep --workspace . BASE..HEAD.

[A terminal recording of the caught lie. The agent reports: Done! Shipped the login endpoint (AUTH1) and the password reset (AUTH2). git log shows one commit — AUTH1: ship the login endpoint. dos verify AUTH AUTH1 answers SHIPPED (exit 0); dos verify AUTH AUTH2 answers NOT_SHIPPED via none (exit 1) — caught. The exit code is the verdict: gate the agent's done on it and a false claim cannot land.]
The whole pitch in one recording: the agent claims two features shipped; git backs one. dos verify answers from the commits, the lie exits 1, and a gate on that exit code refuses the false "done". Every line is the real CLI's verbatim output — scripts/build_caught_lie_cast.py re-records it whenever the output changes.

[Two agent fleets side by side. Left, no referee: agents all report 'done!', every report is believed, and silent corruption (lies, collisions, spin) piles up into a codebase that 'sorta works' and can't be changed. Right, DOS adjudicates: dos verify reads git and the run branches to SHIPPED (exit 0, land it) or NOT_SHIPPED (exit 1, re-dispatch — caught), and that verdict steers the next step.]
Run a fleet of agents on one repo. The left loop just feels like progress; the right one you can steer. The only difference is a verdict DOS reads from the real world — here, git — never the agent's word.

An AI agent will tell you it finished. DOS checks the real world instead of taking its word — and the nearest piece of the real world is your git history. An agent says it shipped the login endpoint; did it? Run one command, dos verify, and it answers from the artifacts the work left behind, not from what the agent typed: a commit backs the claim → SHIPPED, exit 0; nothing landed → NOT_SHIPPED, exit 1. The agent's story never enters into it. (Git is just the first witness DOS reads; the file tree, the clock, a CI status, a test environment's own state are others — anything the agent didn't author.)

dos verify AUTH AUTH1   # → SHIPPED      AUTH AUTH1 e62f74d   (exit 0)dos verify AUTH AUTH2   # → NOT_SHIPPED  AUTH AUTH2           (exit 1)

That's the smallest version. It scales up, too: point a dozen agents at one repo — in CI, in a fleet, racing on the same files — and DOS also tells you which ones are stepping on each other, which one is spinning in circles, and which claim of "done" is real. Every answer comes from the artifacts (git, the file tree, the clock), never the narration. It works on a plain git repo with zero config and gets smarter the more you tell it, and the only thing you ever install is one small Python package.

Setup — use DOS in your repo

DOS is a Python package you install into the existing git repo where your agents work. You do not need to clone or fork the DOS source unless you plan to develop DOS itself.

cd path/to/your-repopip install dos-kerneldos init --hooks auto   # detects the agent runtime(s) already in this repodos doctor              # shows exactly what this workspace is using

From then on, your agent cannot tell you "done" unless the work actually landed, two agents cannot silently overwrite each other's files, and a run that stalls gets flagged instead of quietly spinning. dos init prints the config and hook entries it wrote; removing those generated entries undoes the setup. If no runtime is detected, it says so and lists the explicit choices — it never guesses.

_{v0.30.0 · 5,600+ tests · CI: Python 3.11–3.13 on Linux + a Windows 3.13 smoke run · the only runtime dependency is PyYAML · MIT.}

🧭 Where to go next: the why & evidence (plain-words story, the 20-lines-of-bash answer, what's proven), wire it into your stack (MCP · hooks · install), the syscall + CLI reference, or, reading this as an AI agent?, AGENTS.md — build/test/check in three lines. The full map is the router just below.

🔤 Five words the rest of this page leans on. A plan is a named goal (AUTH); a phase is one shippable step of it (AUTH1); a lane is the slice of the file tree one agent may touch; the oracle is the part of DOS that reads the evidence and rules; a stamp is the mark a shipped phase leaves in a commit subject (AUTH1: …) — the thing the oracle greps for. That's the whole vocabulary.

In plain words

A coding agent does work, then tells you how it went. Usually the story is true; sometimes it's the cheerful "all work completed!" from a worker that shipped nothing. With one agent you catch that yourself by re-reading its output — a real tax you already pay. Run twenty at once and that tax stops being payable: nobody reads everything, each worker grades its own homework, and the unchecked problems pile up quietly until the codebase sorta works and nobody can safely change it. DOS is the referee that never reads the story — it reads what happened (the commit, the file, the clock) and hands you a verdict no narration can move. It costs about an afternoon, has one runtime dependency, and stays in its lane: it tells you what happened, never whether the code is good — quality stays with your tests and reviews. (The full plain-words version.)

Measured, not asserted

Every number here is scored against a fact the agent can't fake (a test environment's DB state, git history). A DOS gate caught 15 "I shipped it" lies in 258 tasks across two models with zero false alarms; the same referee stopped 6 of 8 silent collisions on one shared record; quitting doomed runs at the right moment saved ~11% of fleet compute with 0 of 1,634 winners wrongly killed; and the reward-set admission label lifted acceptance precision 60% → 100% by purging poison a self-graded collector keeps. The methodology, the two money-moment figures, and the projected-vs-bet honesty gradient are in what's proven and what's still a bet.

Where the rest of the docs are

This page keeps the hook, the demo, and the failure it fixes. Everything deeper lives on a focused page — find the question you arrived with and jump:

Optional demo — try it in 60 seconds

Got a terminal? This runs the whole thing in a throwaway repo — one command scaffolds it, makes a real commit, verifies it, and cleans up after itself:

pip install dos-kernel      # PyYAML is the only runtime depdos quickstart              # → SHIPPED AUTH AUTH1 … then NOT_SHIPPED AUTH AUTH2

One SHIPPED, one NOT_SHIPPED: the first is a claim git can back, the second is a claim nothing landed for. That contrast is the product. The demo closes with a router to wherever you already run agents — a Claude Code / Cursor tab (dos init --hooks), an MCP host, a CI step, or a fleet — so your next move is one line, not a docs dig. (Add --keep ./demo to keep the repo and poke at it. Don't even want the install? uvx --from dos-kernel dos quickstart runs the same demo ephemerally — nothing left behind.) The same thing by hand, in five lines, is docs/QUICKSTART.md.

[The dos verify money-moment. Two equally-confident agent claims, checked against git. Left, what the agent claims (forgeable): 'Shipped AUTH1 — the login endpoint is done' and 'AUTH2 is done too — all work completed!'. Right, what git actually records: one real commit e389e8b 'AUTH1: ship the login endpoint', and no commit anywhere mentions AUTH2. The two verdicts: dos verify AUTH AUTH1 finds the token in a real commit subject → SHIPPED, exit 0, via grep-subject; dos verify AUTH AUTH2 finds it nowhere → NOT_SHIPPED, exit 1, via none. The confident AUTH2 claim collapses the instant no commit backs it.]
_{Two equally confident claims, one verdict each — SHIPPED for the one git can back, NOT_SHIPPED for the one nothing landed for. Every string is verbatim output of examples/demo/verify_demo.sh. Step through it locally for the click-through version (it's an HTML file — clone the repo and open it in a browser; GitHub shows its source, not the running page).}

The smallest real win: in a CI step or dispatch loop, replace the line that trusts an agent's "done" with dos verify PLAN PHASE and branch on its exit code (0 shipped / 1 not). No parsing, no plan, no config — the CI integration cookbook walks it end-to-end. To run it on a repo shaped like yours, start with Onboard a repo in 10 minutes.

Point the same witness at a review queue when commits pile up faster than anyone can read them. Residual review folds commit-audit's per-commit verdict into three bands — CLEARED (the diff witnessed the claim, so spend ~0 attention re-asking "did it do what it said"), RESIDUAL (a claim git couldn't back — the human's 100%), and the no-claim rest. On this repo's own last 200 commits it cleared 170 of 171 checkable claims: that's the re-review you skip, proven by git rather than a model's confidence score. (CLEARED means the change's shape matched its claim — not that the code is correct; correctness review still applies to every commit. The band can only ever ask for more eyes, never fewer.)

Next level up — wire the verdict into your own stack: Wire it in.

What goes wrong in a fleet

Run a pile of agents at once with nobody refereeing, and here's how it goes: each worker reports its own success, and you believe the reports, because what else is there to go on? The unchecked problems pile up quietly — a lie here, two agents clobbering the same file there, a little scope creep, one worker spinning in circles — until the codebase sorta works and nobody can safely change it.

The trouble is you launched the agents and then let them grade their own homework. DOS gives you the missing signal — a verdict from ground truth — so the loop closes. Here is the same fleet under both regimes:

flowchart LR  subgraph OPEN["NO REFEREE — you believe the narration"]    direction TB    A1["agent: 'done!'"] --> B1[["believed"]]    A2["agent: 'done!'"] --> B1    A3["agent: 'done!'"] --> B1    B1 --> C1["silent corruption piles up<br/>(lies · collisions · spin)"]    C1 --> D1["'sorta works' — can't be changed"]  end  subgraph CLOSED["DOS ADJUDICATES — you steer on a verdict"]    direction TB    A4["agent: 'done!'"] --> V{{"dos verify<br/>reads git"}}    V -->|in git ancestry| S["SHIPPED (exit 0)"]    V -->|found nowhere| N["NOT_SHIPPED (exit 1)"]    S --> L["land it"]    N --> R["re-dispatch / flag — caught"]    R -.verdict steers the loop.-> A4  end

Here are the failures a fleet actually produces, each next to the ground truth that quietly contradicts the worker's story — and the verdict DOS hands back:

The first row is the most common one. The classic tell is a cheerful one-liner, "all work completed!", from a worker that did little or nothing. DOS never reads that line; it reads the ground truth, so the claim collapses the instant no artifact backs it (more in docs/108). That's also what makes it cheap to adopt: verify needs no plan, no registry, no config, and the exit code is the verdict — any shell or CI step can branch on it without parsing a word.

_{Prefer to watch it move? The two loops are also a self-contained animation you step through one frame at a time — clone the repo and open docs/assets/loop_visual.html in a browser. (It's an HTML file, so GitHub shows its source rather than running it — open it locally.)}

Lease scope — single filesystem today. The verification half (verify, commit-audit, liveness) travels across machines freely because it reads git history. The admission half (arbitrate, lane leases) is local-filesystem only: the WAL lives on one disk, and workers on separate machines share no serialization point. A fleet that runs all its workers on one machine or in one shared filesystem is fully covered; a fleet spanning multiple hosts should treat dos arbitrate as advisory (not a hard mutex) until a remote-lease driver ships. See docs/366 for the design.

How far you take it

It works on a plain git init with zero config, and gets smarter the more you tell it. You don't adopt a framework and pick a tier; you start at the shallow end and it keeps paying off as you wade deeper — the same kernel the whole way:

• Zero config. Point dos verify PLAN PHASE at a plain git repo — no plan, no registry, no dos.toml. It answers from commit history alone (via grep-subject / via none). This is the whole of QUICKSTART and the day-one CI win above.
• Tell it your structure. dos init writes a dos.toml (lanes, paths, ship grammar as data); add a plan doc and dos plan lays each phase's claim beside the oracle's verdict. Here's exactly what a plan file looks like (copyable, round-trips with the built-in reader), and four worked example workspaces.
• Teach it your own types. Declare your own block reasons, gate verdicts, output renderers, admission predicates, a model-backed judge, a custom plan dialect, or a whole host driver — all as workspace policy, never a fork. The map is docs/HACKING.md (seven extension axes) + the copy-me examples/dos_ext/.

How you plug it in

That slope is how deep your config goes. The other axis is how you call the referee at all — and you adopt through whichever surface matches how you already work, not by restructuring your stack. The same kernel verdicts are reachable through every row here, lowest-friction first:

The two axes are independent: a zero-config repo can adopt through any surface, and a deeply-configured one still answers over the same CLI and MCP tools. Start at the top row — it's the one that costs nothing to try. The first two rows also compose: MCP advises (the agent checks its own work), hooks enforce (the host stops a bad action) — wire both for the full loop.

Those surfaces are the upstream half of the value chain — who calls the referee. The same verdicts also flow downstream, to the systems that act on them: every adjudication lands in a verdict journal that dos export drains to your observability stack (Datadog / Honeycomb / Grafana — docs/266), dos notify pushes what-needs-a-human to Slack, dos reward gates what a fine-tune may train on, and dos attest mints a signed receipt a skeptic can check without loop access (docs/246). One kernel, one verdict vocabulary, from the agent's tool call to your dashboard.

Next level up — run it every day: Operating a fleet.

From the same team

DOS is one of three open tools from Anthony Chaudhary for running AI agents you can actually trust — at three different moments:

• fak — the agent kernel — DOS reads what an agent already did (after the fact, from git and other witnesses it can't forge); fak governs what an agent is allowed to do as it happens. A single static Go binary that fronts your token engine and adjudicates every tool call at the boundary — capability gate, tool-result quarantine, audit trail — the inline gate to DOS's out-of-loop referee. go install github.com/anthony-chaudhary/fak/cmd/fak@latest · docs.
• Diffgram — the AI datastore for human supervision of AI data (labeling, workflow, catalog). Where DOS and fak supervise the agents, Diffgram supervises the data they learn from and produce.

Citation

The ideas here are written up in a paper — "Verification Is All You Need — But Not Where You Think" — on the out-of-loop referee for agent fleets. A built PDF lives at paper/releases/; the arXiv preprint is in preparation. Until the arXiv ID lands, cite the repository:

@misc{dos_kernel,  title        = {Verification Is All You Need --- But Not Where You Think},  author       = {Chaudhary, Anthony},  howpublished = {\url{https://github.com/anthony-chaudhary/dos-kernel}},  note         = {DOS --- the Dispatch Operating System; arXiv preprint in preparation},  year         = {2026}}

License

MIT — see LICENSE.