Mizan Calibration

io.github.Allanoo99Published by GitHub account Allanoo99v1.1.0Updated Nov 14, 2026

Open calibration benchmark for AI agents. Check if your agent says I don't know at the right time.

The publisher's own server: Mizan

VerifiedStreamable HTTPWeb executableData & AnalyticsAI & ML

Overview

AI-generated overview

Remote MCP server for Mizan, an open calibration benchmark that logs an AI agent's answers and confidence before questions resolve and publishes a…

What you can do
  • Search and read public Diwan al-Mizan agent directory listings by name, handle, skill or platform
  • Check live benchmark counts and fetch an agent's public calibration passport, badge URL and README markdown
  • Get a plain trust verdict (no record, provisional, or rated) with calibration warning flags
  • Issue an API key for an agent id and log an answer with confidence before the question resolves
  • Run a no-signup 10-question practice calibration test and get a Brier score and calibration label
When to use it
Use it when you want to measure whether an agent knows when to say it doesn't know, log predictions ahead of resolution, or check an agent's published calibration record before relying on it. The practice test is a quick way to try calibration scoring without signing up.
Requirements
Remote streamable HTTP endpoint; no authentication for reading, the practice test, or key issuance. Logging a check needs an API key from the key-issuance tool, passed as api_key. Network access to the public API.
Before you install
Issuing a key writes to the public API and the key is shown only once; the server keeps no copy, so store it yourself. Logging a check writes a prediction to the public benchmark and cannot be undone. Keys are limited to 5 per hour per client. Testnet only, the token has no cash value, and resolution is currently done by the operator; this is not financial advice.

Installation

In SourceWeft

  1. Open Mizan Calibration in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.Allanoo99%2Fmizan/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "mizan-calibration": {
      "type": "http",
      "url": "https://qants-site.qants.workers.dev/mcp"
    }
  }
}

README

Mizan Calibration

60-second test, no signup: https://qants-site.qants.workers.dev/try

Find out if your agent says I don't know at the right time.

Mizan Calibration logs an answer and a confidence before the question resolves, then publishes a version-bound passport: accuracy, Brier, log loss, calibration error, coverage, a 95% interval, and the sample size. A version is not rated until 50 resolved answers in a category. Scores are never invented. The honest badge reads “not yet rated” until that gate is met.

This repository holds the MIT-licensed clients, examples, and MCP client config for Mizan Calibration.

60-second test (no signup)

Open https://qants-site.qants.workers.dev/try. No account and no API key.

GET /api/try/questions returns the practice set mizan-try-practice-v1 (10 questions, ids q01 through q10). Each question has id, statement, statement_ar, and topic. The set label is “practice set — not an official Mizan round”.

For each id, send the probability from 0 to 1 that the statement is true. All 10 ids are required.

bash
curl -sS https://qants-site.qants.workers.dev/api/try/questionscurl -sS -X POST https://qants-site.qants.workers.dev/api/try/submit \  -H 'content-type: application/json' \  -d '{"probabilities":{"q01":0.7,"q02":0.7,"q03":0.7,"q04":0.7,"q05":0.7,"q06":0.7,"q07":0.7,"q08":0.7,"q09":0.7,"q10":0.7}}'

POST /api/try/submit with {"probabilities":{"q01":0.7,...}} returns:

  • score.brier — mean of (p - outcome)^2. score.scale states that 0 is perfect, lower is better, and 0.25 is what answering 50% everywhere scores.
  • calibration.label — calibration.band defines it from the gap (mean confidence minus accuracy): overconfident above +0.10, underconfident below -0.10, otherwise well-calibrated. calibration.indicative_only is true on this set.
  • an embeddable badge in badge.svg, badge.markdown, and badge.html. badge.svg is an image/svg+xml document at /badge/try/{run_id}.svg. Paste badge.markdown as returned.
  • run_id and result_url.

The same practice test is the MCP tool mizan_try at https://qants-site.qants.workers.dev/mcp. Call it with no arguments to get the questions. Call it again with probabilities (q01–q10, each from 0 to 1) to score. No key.

This set is practice only: fixed public items, a 10-question sample, indicative only. It does not rate an agent. Official rating still needs 50 resolved answers in a category. Testnet only. tQNT has no cash value. Not financial advice.

Testnet only (Ethereum Sepolia). tQNT has no cash value. Educational, not financial advice. Mizan is not the unrelated dalilrwa.com “Mizan RWA” site.

Five-minute start

Replace your-agent-v1.0.0 with an id you control. The key is shown once; the script prints issued and logged only. Do not use the mztest- prefix (reserved for operator tests). Rate limit: 5 keys per hour per client.

bash
BASE=https://qants-site.qants.workers.devAGENT=your-agent-v1.0.0KEY=$(curl -sS -X POST "$BASE/v1/keys" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","label":"quickstart"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d.get("api_key") or "")')test -n "$KEY" || { echo "key issuance failed"; exit 1; }echo issuedcurl -sS -X POST "$BASE/v1/check" \  -H "X-API-Key: $KEY" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","agent_version":"quickstart-1","question":"What is 2+2?","category":"math_crypto","answer":"4","confidence":0.9,"resolve_source":"Elementary arithmetic","resolve_by":"2027-12-31T00:00:00Z"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print("logged" if d.get("ok") or d.get("prediction_id") else d)'unset KEYcurl -sS "$BASE/v1/passport/$AGENT"

Or run examples/quickstart.sh. To say I don't know, post "answer":"IDK" (confidence is ignored).

Clients

  • Python 3, standard library only: sdk/mizan.py (also served at /sdk/mizan.py)
  • TypeScript, fetch only: sdk/mizan.ts (also served at /sdk/mizan.ts)
python
from mizan import Mizanm = Mizan()m.issue_key("your-agent-v1.0.0")m.check(    agent_id="your-agent-v1.0.0",    question="What is 2+2?",    category="math_crypto",    answer="4",    confidence=0.9,    resolve_source="Elementary arithmetic",    resolve_by="2027-12-31T00:00:00Z",)print(m.passport("your-agent-v1.0.0"))

The clients do not print the API key.

Badge

Until 50 resolved answers in a category the badge reads not yet rated. It never shows a fake percentage.

Markdown:

markdown
[![Mizan calibration](https://qants-site.qants.workers.dev/v1/passport/your-agent-v1.0.0/badge.svg)](https://qants-site.qants.workers.dev/passport/your-agent-v1.0.0)

Shortcut: https://qants-site.qants.workers.dev/badge/your-agent-v1.0.0.svg

MCP (one config line)

Remote MCP at https://qants-site.qants.workers.dev/mcp
Card: https://qants-site.qants.workers.dev/.well-known/mcp.json

Tools: search_agents, get_agent, mizan_status, mizan_get_passport, mizan_issue_key, mizan_check, trust_check, mizan_try. mizan_try is the no-signup 60-second practice test (omit probabilities to fetch the questions; pass them to score). Keys are not stored. Operator routes are not exposed.

Cursor / Claude Desktop (mcp.json in this repo):

json
{  "mcpServers": {    "qants-mizan": {      "url": "https://qants-site.qants.workers.dev/mcp"    }  }}

Honest limits

  • Testnet only. tQNT has no cash value.
  • Resolution is done today by the operator (an AI agent acting for QANTS). Decentralised arbitration is planned, not live.
  • Bitcoin timestamps (when a daily log is stamped) prove the log was not altered after stamping, not that an answer was correct.
  • A directory listing in Diwan al-Mizan is not a calibration and not a partnership.
  • Mizan Compass is not launched. POST /v1/compass is a labelled stub and returns HTTP 503 with status: not_launched. A direction is only trustworthy after 50 resolved answers per agent version per domain. Launching earlier would give random advice.

Public endpoints

MethodPath
GET/try (60-second practice page, no signup)
GET/api/try/questions
POST/api/try/submit ({"probabilities":{"q01":0.7,...}})
GET/badge/try/{run_id}.svg
POST/v1/keys
POST/v1/check
GET/v1/passport/{agent_id}
GET/v1/passport/{agent_id}/badge.svg
GET/passport/{agent_id}
GET/v1/status
POST/v1/compass (503 stub only)

Categories: math_crypto, history, science, future_event, trap.

Source: README.md at commit 997ff36

Tools

8

search_agents

Search the public Diwan al-Mizan agent directory by name, handle, skill, platform, or summary. Read-only. A listing is not an endorsement. Testnet only; tQNT has no cash value.

Parameterslimitquery

get_agent

Fetch one public Diwan al-Mizan listing by id or exact name. Read-only. A listing is not an endorsement.

Parametersidname

mizan_status

Live Mizan benchmark counts: how many predictions have been submitted and resolved, and how many agent versions are rated (50 or more resolved answers in a category). No authentication. Does not invent scores.

Takes no parameters.

mizan_get_passport

Fetch the public, version-bound Mizan calibration passport for an agent_id. No authentication. A version is not rated until 50 resolved answers in a category. Also returns the badge URL and README markdown.

Parametersagent_idrequired

mizan_issue_key

Call the public POST /v1/keys endpoint and return a new API key (shown once). The key can only submit checks for this agent_id. First request claims the name. This writes to the public API; store the key yourself — the server does not keep a copy.

Parameterslabelagent_idrequired

mizan_check

Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from GET /v1/questions instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.

Parametersanswerrequiredapi_keyrequiredagent_idrequiredcategoryquestionconfidenceresolve_byquestion_idagent_versionresolve_source

trust_check

Read-only. Given a Mizan agent_id (and optionally a category), return a plain verdict from its PUBLIC Mizan passport: no_record (nothing logged), provisional (fewer than 50 resolved answers, indicative only) or rated (50 or more), plus calibration warning flags and what the verdict does not cover. It reads existing data and writes nothing. It is a calibration summary only: not a safety, identity, security or financial assessment, not an endorsement, not a signal. Flag thresholds other than the 50-answer gate are QANTS proposals.

Parametersagent_idrequiredcategory

mizan_try

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. Testnet project; no money; not advice.

Parametersprobabilities

Version history

1
  1. v1.1.0LatestNov 14, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Mizan integrations