Evidrift

Evidrift — replay-verified JSON reductions and API drift evidence

English | 繁體中文

[CI] [npm version] [Website] [Indexed on TensorBlock MCP Index] [bm1016bm-svg/evidrift MCP server]

Failures arrive noisy. APIs drift quietly. Evidrift turns both into deterministic evidence.

Evidrift has two deliberately separate workflows:

• ReproMin repeatedly removes JSON data, replays each candidate against a disposable local HTTP target, and keeps only reductions that still match the selected failure identity.
• Contract drift records an exact TypeScript call signature or repository JSON value as a content-addressed Receipt, then makes CI recompute it before merge.

Local-first CLI. STDIO MCP server for contract recording only. No account, cloud backend, telemetry, or LLM judge.

[Evidrift — AI dependency lockfile]

[Real Evidrift CLI demo: a dependency contract passes, its TypeScript signature changes, and Evidrift catches the drift before merge]

The animation is rendered from a captured CLI transcript. The PASS, changed signatures, affected file, and deterministic FAIL come from an actual local evidrift demo run; only the scene headings are editorial.

Quick Start — Minimize a Failing Request

Requires Node.js 22 or newer. The zero-setup demo starts a disposable loopback server, verifies one HTTP failure, minimizes its JSON request, verifies the result again, and shuts the server down:

npx --yes evidrift@latest repro-demo

The result is not selected by an LLM. Every accepted reduction is actually replayed and must match both the expected HTTP status and error identity.

For your own local endpoint:

npx evidrift minimize \  --request failing-request.json \  --status 500 \  --response-pointer /error/code \  --response-equals '"INVALID_FILTER"' \  --output minimal-repro.json \  --confirm-replay yes

Then replay the content-addressed artifact once:

npx evidrift reproduce --artifact minimal-repro.json --confirm-replay yes

See the complete ReproMin fixture, guarantees, and replay safety boundary.

Quick Start — See Drift in One Command

Requires Node.js 22 or newer. Nothing to install globally:

npx --yes evidrift@latest demo

The command creates a disposable local fixture, records the optional options parameter on parseConfig, checks it successfully, changes the fixture so options is required, then proves that evidrift check catches the mismatch. It runs no downloaded package code.

If that is a failure you want caught before merge, star Evidrift on GitHub.

Real-world Lab — React 19 useRef

The synthetic demo is fast; this source-repository lab proves the same workflow against a real, documented ecosystem change. React's official upgrade guide says React 19 makes the first useRef argument required. Evidrift records the no-argument overload from @types/[email protected], upgrades only that type package to @types/[email protected], and reports the missing overload:

git clone https://github.com/bm1016bm-svg/evidrift.gitcd evidriftnpm ci --ignore-scriptsnpm run demo:react-19

Both package versions are exact and verified after installation. The lab disables dependency lifecycle scripts, and its final evidrift check must return a deterministic contract mismatch. See the reproduction and complete trust boundary and the official React 19 upgrade guide.

Supported Today

Installation — Add It to a Repository

Initialize the current repository without a global install, account, API key, or cloud backend:

npx --yes evidrift@latest init

To add an idempotent pull-request workflow with file-and-line annotations:

npx --yes evidrift@latest init --github-actions

The initializer detects npm, pnpm, or Yarn, adds scripts.evidrift:check, and creates .github/workflows/evidrift.yml. Existing scripts and workflows are reported and preserved instead of being overwritten.

To pin Evidrift for a team or CI workflow:

npm install --save-dev evidriftnpx evidrift init

That is the product: make an AI assumption reviewable now, then make CI check the same contract later.

Use It in a Repository

The dependency must already be installed inside the target repository, and the affected code path must name a real file.

cd /path/to/your/repositoryevidrift init
evidrift record \  --project . \  --package your-package \  --symbol exportedFunction \  --parameter options \  --claim "exportedFunction accepts the options used here." \  --code src/caller.ts:12
evidrift check

When --code includes a line containing an overloaded call, Evidrift asks TypeScript which overload that call actually resolves to. --overload <number> remains an explicit fallback for incomplete or non-compiling call sites.

Lock one value from a repository-local OpenAPI or JSON Schema document:

evidrift record \  --json openapi.json \  --pointer /paths/~1users/get/operationId \  --claim "The generated client calls listUsers." \  --code src/client.ts:24

JSON Pointer follows RFC 6901, including ~1 for / and ~0 for ~. Evidrift reads .json files only; it never fetches URLs or resolves remote references.

Coding agents call the same core through evidrift_record and evidrift_record_json_pointer. Minimal Codex, Claude Code, and Cursor setup is included.

How It Works

flowchart LR  Author["Developer"] --> Minimize["Minimize failing JSON"]  Minimize --> Replay["Replay loopback failure"]  Replay --> Repro["Content-addressed reproduction"]  Agent["Coding agent or developer"] --> Record["Record one assumption"]  Record --> Adapter["TypeScript or JSON adapter"]  Adapter --> Receipt["Content-addressed Receipt"]  Receipt --> Review["Git review"]  Review --> Check["evidrift check in CI"]  Check --> Result{"Contract still matches?"}  Result -->|Yes| Pass["PASS"]  Result -->|Source unavailable| Warn["WARNING"]  Result -->|No or tampered| Fail["FAIL"]

The CLI and MCP server are thin entry points over the same core. The complete component map, check policy, resource bounds, and trust boundary are documented in Architecture.

The Files

Evidrift writes one lock and one immutable JSON file per Receipt:

.evidrift/  evidence.lock  receipts/    <64-character-sha256>.json

There is no .evidrift/receipts.json. evidence.lock contains only content-addressed Receipt IDs:

{  "receipts": ["sha256:9bfbb065cff372abe52e8e269123959e9f2ae84cd02230dc751f768ac5e4c274"],  "schemaVersion": 1}

Each Receipt stores the claim and affected code plus one deterministic contract: an installed TypeScript symbol signature, or a repository JSON path, pointer, canonical value, and hashes. See the Receipt schema.

Add It to CI

Generate the package script and read-only pull-request workflow:

npx evidrift init --github-actions

The generated package script pins the current Evidrift release:

{  "scripts": {    "evidrift:check": "npx --yes [email protected] check"  }}

Run the same pinned check locally with npm run evidrift:check.

The generated workflow installs locked repository dependencies with lifecycle scripts disabled, then runs the Marketplace Action. Contract mismatches and integrity failures become PR errors at the affected file and line; source-change and unavailable-evidence results become warnings.

evidrift check --annotations github

With --format json, GitHub workflow commands are written to standard error so standard output remains valid JSON. The complete GitHub Actions setup documents the generated workflow, permissions, package-manager behavior, and manual setup.

The root action.yml provides a Marketplace-ready contract check. It runs only the network-free contract workflow; ReproMin replay is never triggered by that Action.

CI Behavior

evidrift check does not trust saved matched or verified flags. It validates the Receipt, reloads the source, and recomputes the selected signature or JSON value.

For CI systems, coding agents, and other tools, request a versioned JSON report without changing those exit codes:

npx evidrift check --format json

The report includes schemaVersion, the Evidrift version, summary counts, and the ordered CheckResult objects. It contains no timestamp or absolute repository root, so equal results serialize identically. See the JSON check report contract.

A one-line manual edit to a Receipt produces an actionable integrity report:

FAIL evidence_integrity sha256:...Message: Receipt content hash mismatch.Receipt ID: sha256:...Action: Do not trust or hand-edit this Receipt. Restore it from version control, or intentionally create a new Receipt with `evidrift record`.

The project workflow runs the full gate on Linux and Windows with Node.js 22 and 24. Third-party Actions are pinned to full commit SHAs.

In a human TTY, check, diff, explain, and demo use a spinner plus green ✅, yellow ⚠, and red ❌ status output. Redirected output, CI, TERM=dumb, and NO_COLOR stay ANSI-free and keep the stable plain-text format used by agents and tests.

Why This Is Not RAG, Sonar, or AI Review

Use all of them if they help. Evidrift covers one gap: the reason code was written can go stale even when the code itself did not change.

Frequently Asked Questions

What is API drift?

API drift is a change to a dependency or contract after code was written against it. Evidrift checks two deterministic forms: the TypeScript call signature selected at an affected code location, and a canonical value selected from repository-local OpenAPI JSON or JSON Schema.

Is Evidrift a contract-testing tool?

It is narrower than end-to-end contract testing. Contract tests exercise provider and consumer behavior; Evidrift locks one explicit assumption that influenced code and revalidates that assumption in CI without running dependency code.

Does Evidrift work with Codex, Claude Code, and Cursor?

Yes. They can call the local STDIO MCP server to create Receipts through the shared verification core. The agent cannot directly mark a Receipt as verified. See the minimal MCP configurations.

Does Evidrift fetch OpenAPI URLs or execute package code?

The contract adapters do not. They inspect installed TypeScript declarations and repository-local .json files without fetching URLs, resolving remote $ref, importing dependency JavaScript, or executing arbitrary commands.

minimize and reproduce are separate, explicit replay commands. They send JSON only to literal loopback HTTP addresses after --confirm-replay yes; they never run during check or through the MCP tools.

Does Evidrift prove that AI-generated code is correct?

No. It detects deterministic evidence drift and Receipt tampering. It does not prove runtime correctness, validate free-text semantics, or eliminate hallucinations. See What Evidrift Does Not Prove.

CLI

evidrift init [--github-actions]evidrift record --project <path> --package <name> --symbol <name> \  [--parameter <name>] [--overload <number>] --claim <text> --code <path[:line]>evidrift record --json <path> --pointer <RFC6901> --claim <text> --code <path[:line]>evidrift check [--format text|json] [--annotations none|github]evidrift diffevidrift explain <receipt-id>evidrift demoevidrift repro-demoevidrift minimize --request <json> --status <code> \  (--response-contains <text> | --response-pointer <RFC6901> --response-equals <json>) \  --output <json> --confirm-replay yesevidrift reproduce --artifact <json> --confirm-replay yesevidrift mcp

Repository-scoped commands accept --root <repo>. record requires an initialized .evidrift/evidence.lock; repro-demo and mcp do not use a repository root.

evidrift mcp starts the same local STDIO server exposed by the evidrift-mcp bin. It exists so package registries and MCP clients can launch Evidrift deterministically from the main npm package.

Trust Boundary

.evidrift/receipts/*.json is untrusted input. Every check:

1. Strictly validates lock and Receipt schemas.
2. Derives file paths only from full SHA-256 IDs.
3. Recomputes the expected contract hash and Receipt content hash.
4. Resolves declarations or repository-local JSON without importing package JavaScript, running shell commands, making network requests, or calling an LLM.
5. Reports evidence integrity, source drift, semantic support, and runtime correctness separately.

The parser refuses more than 1,024 Receipt IDs or 64 call signatures per symbol. TypeScript evidence is confined to repository files and capped at 256 source files, 2 MiB per file, and 16 MiB total. JSON sources are capped at 4 MiB and selected canonical values at 1 MiB. Dynamic text is rejected or escaped so a Receipt cannot inject terminal controls or fake CI lines.

Content hashes detect inconsistent edits; they do not prove authorship. Someone who rewrites a Receipt, recalculates its ID, and changes evidence.lock can create new internally valid evidence. Git review and branch protection must catch that replacement. See Architecture.

ReproMin has a separate boundary because replay can cause side effects. It accepts only literal loopback HTTP targets, refuses redirects and common secret-shaped input, requires explicit replay confirmation, caps requests and response inspection, and never runs from check or the MCP server. The artifact records what Evidrift observed; its content hash detects inconsistent edits but does not authenticate probe history. reproduce verifies one current replay.

What Evidrift Does Not Prove

Evidrift does not prove code is correct. The contract workflow does not prove a free-text claim, execute runtime behavior, eliminate hallucinations, scan dependency vulnerabilities, or validate arbitrary URLs.

ReproMin does not prove a global minimum or identify the root cause. Without an exhausted budget it establishes only 1-minimality under its documented JSON reducers and the chosen failure predicate. A weak predicate can still describe the wrong bug, which is why status alone is rejected.

The contract workflow resolves overloaded TypeScript calls from the affected path:line when the consumer code compiles and TypeScript can identify one declared overload. Invalid, ambiguous, or missing calls are refused rather than guessed; --overload is the explicit fallback. The Receipt stores the selected normalized signature and hash, so declaration reordering does not cause false drift.

The json.pointer adapter locks canonical values in repository-local .json files. It does not support YAML, URLs, remote $ref, schema validation, or semantic equivalence. It follows repository-local TypeScript declaration imports, but does not expand every named type into a deep structural contract. Missing or unreadable source is a visible but non-blocking warning. These are deliberate limits, not hidden guarantees.

The runnable boss-fight test resolves one of three overloads from a real call using a complex cross-file type alias, survives declaration reordering, and deterministically fails when only the selected overload changes.

Development and UAT

npm run format:checknpm run lintnpm run typechecknpm testnpm run uatnpm run check

npm run verify runs the release gate. Tests use temporary fixtures and disposable loopback servers; they require no secrets, paid APIs, Evidrift backend, or external network access. The detailed UAT report maps each acceptance case to an automated test and states the remaining risks.

License

Mozilla Public License 2.0.