Agent evidence

io.github.probityaiPublished by GitHub account probityaiv0.2.5Updated Nov 3, 2026

Offline, read-only tools that verify AI claims against evidence and replay conformance vectors.

VerifiedSTDIODesktop onlyDeveloper ToolsSecurity & MonitoringAI & ML

Overview

AI-generated overview

Offline, read-only MCP server that lets an assistant verify claims about AI systems against supplied evidence bytes and replay conformance vectors.

What you can do
  • Verify a single claim against caller-supplied evidence bytes, or run a bundled sample claim that returns supported, contradicted or not_established.
  • List shipped vector corpora with suite revisions and vector counts, replay a corpus with the reference verifier in paged rows, and explain one vector's manifest entry, conditions and input bytes.
  • Look up terms in the released vocabulary by exact key, with close-key suggestions.
  • Check that an artifact has a digest named in a TEA, CycloneDX or in-toto document.
  • Canonicalize a JSON document to RFC 8785 bytes and verify a DSSE envelope against caller-trusted Ed25519 keys.
When to use it
Useful for auditors, incident reviewers and relying parties who need a replayable decision on an AI claim, and for tool authors who want their records checked the same way. It suits offline review work where evidence is already pinned locally.
Requirements
Runs as a local stdio process installed from the PyPI package agent-evidence-mcp. The README's quick start needs Python 3.11 or newer and uv, and the verifier package probity-verify. No authentication, environment variables or headers are declared.
Before you install
Read-only and offline per its instructions: nothing is fetched or written. Decisions depend entirely on the evidence bytes the caller supplies, so a supported result is only as trustworthy as that input. DSSE verification trusts only the Ed25519 keys the caller provides.

Installation

In SourceWeft

  1. Open Agent evidence in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

Probity

[CI] [PyPI] [Downloads] [Licence] [Stars]

Open tools that check claims about AI agents against evidence you pinned: an offline verifier today, with the observer joining this repository. The standard layer (corpus, vocabulary, admission rules and crates) lives in https://github.com/probityai/agent-evidence-vectors.

[A report of seven passages with one missing. A log reader reads what the report holds, passes over the gap and ends in a check. probity-verify reads the report against the source capture, stops at the gap and returns contradicted, reason selected_span_missing_from_record.]

It's for auditors, incident reviewers and relying parties who need a decision they can replay, and for tool authors who want their records checked the same way.

Install the released verifier: uv add probity-verify==0.2.4, then pip install probity-verify==0.2.4.

Verified by Probity (github.com/probityai/probity). The probityai command line is released: uv add probityai==0.1.0, then pip install probityai==0.1.0.

Quick start

Check a sample claim with the released package. You need Python 3.11 or newer and uv; there is nothing to clone:

sh
samples="$(uvx --from probity-verify==0.2.4 python -c 'import importlib.resources as r; print(r.files("probity_verify") / "samples")')"uvx probity-verify==0.2.4 "$samples/case.json" --policy "$samples/policy-june.json"

It prints a decision packet that opens:

text
Probity decision packet v1Case: "source-coverage-demo"Claim: source_text_coverage/v1Decision: contradictedReason: selected_span_missing_from_record

A selected passage is in the source capture but missing from the stored report. Swap in policy-april.json for not_established, or case-complete.json with policy-complete.json for supported.

Status

memberwhat it isreleased
packages/probity-verifyoffline verifier: supported, contradicted or not_established for a claim against pinned bytes0.2.2 on PyPI
packages/observerrecords what an agent run did into pinned evidence the verifier can check0.1.1 on PyPI (agent-evidence-observer)
packages/ratchetsigned, replayable receipts for improving an agent against held-out cases0.3.2 on PyPI (probity-ratchet)
packages/probityaithe probityai command; probityai verify today, each member adds its own subcommand0.1.0 on PyPI (probityai)

Work on the repository

sh
uv sync --all-packages --all-extrasuv run python -m pytest

Documentation

pageread it for
probity-verifythe verifier, its adapters and its recipes
Repository layoutthe workspace, the probityai command seam, CI and releases
Agent instructionscommands to run before a change
Contributingreporting a problem and opening a pull request

License

Apache-2.0, per directory: each member carries its own LICENSE.

Source: README.md at commit 15b0b4b

Tools

9

verify_claim

Verify a claim against evidence

Decide one claim against evidence bytes the caller supplies, with probity-verify.

Parameterscaserequiredpolicyrequiredartifacts

verify_sample

Run a bundled sample claim

Run a bundled sample claim: contradicted, not_established or supported.

Parametersscenariorequired

list_vector_corpora

List conformance vector corpora

Name every shipped corpus with its suite revision and vector counts where it declares them.

Takes no parameters.

run_vectors

Replay a conformance vector corpus

Replay one corpus with the bundled reference verifier; per-vector rows come back in pages.

Parameterslimitcorpusoffset

explain_vector

Explain one conformance vector

Return one vector: its manifest entry, the conditions it tests, and its input bytes.

Parameterscorpusvector_idrequired

lookup_term

Look up a vocabulary term

Look a term up in the released vocabulary by exact key; otherwise suggest close keys.

Parameterstermrequired

check_artifact_digest

Check an artifact digest against a document

Check that an artifact has a digest a TEA, CycloneDX or in-toto document names.

Parametersdocumentrequiredartifact_textartifact_base64

canonicalize_json

Canonicalize JSON under RFC 8785

Admit one JSON document and return its RFC 8785 canonical bytes, with the jcs-admit crate.

Parametersprofiledocumentrequired

verify_dsse

Verify a DSSE envelope

Verify a DSSE envelope against Ed25519 keys the caller trusts, with the dsse crate.

Parametersenveloperequiredthresholdtrusted_keysrequired

Version history

3
  1. v0.2.5LatestNov 1, 2026
  2. v0.2.4Oct 29, 2026
  3. v0.2.0Oct 20, 2026

Score

90% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts
  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts