Mizan Calibration

io.github.Allanoo99由 GitHub 帳號 Allanoo99 發布v1.1.0更新於 Nov 14, 2026

Open calibration benchmark for AI agents. Check if your agent says I don't know at the right time.

發布方自己的服務:Mizan

已驗證Streamable HTTP可網頁執行資料與分析AI 與機器學習

概覽

AI 產生的概覽

Mizan 校準基準的遠端 MCP 伺服器:在問題揭曉前記錄 AI 代理的答案與信心度,並發布綁定版本的校準護照。

可以做什麼
  • 依名稱、帳號、技能或平台搜尋並讀取 Diwan al-Mizan 公開代理目錄項目
  • 查看即時基準計數,取得代理的公開校準護照、徽章網址與 README 文字
  • 給出簡單信任結論(無紀錄、暫定或已評級)及校準警示標記
  • 為代理 id 簽發 API 金鑰,並在問題揭曉前記錄答案與信心度
  • 執行免註冊的 10 題練習校準測驗,取得 Brier 分數與校準標籤
適用情境
適合用來衡量代理是否在適當時機承認不知道、在問題揭曉前記錄預測,或在依賴某個代理前查看其公開校準紀錄。練習測驗可在不註冊的情況下快速體驗校準評分。
執行需求
遠端 streamable HTTP 端點;讀取、練習測驗與簽發金鑰皆無需驗證。記錄檢查需要由簽發工具取得的 API 金鑰,以 api_key 傳入。需要能連線至該公開 API 的網路。
安裝前請注意
簽發金鑰會寫入公開 API,金鑰僅顯示一次,伺服器不保留副本,請自行妥善保存。記錄檢查會把預測寫入公開基準且無法撤銷。金鑰限制為每用戶端每小時 5 個。僅限測試網,代幣無現金價值,目前由營運方完成判定;不構成財務建議。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 Mizan Calibration,將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Web executable,透過 Streamable HTTP。 遠端服務在工作區中設定後即可從網頁執行環境執行。

交給代理程式安裝

把這段提示詞貼上到 Claude Code、Codex、Cursor 等代理程式,或 SourceWeft 對話中。代理程式會讀取該伺服器的安裝指南,先向你展示發布者、端點和登入方式,經你同意後再把它新增到你的客戶端。金鑰不會寫進任何命令。

閱讀 https://sourceweft.com/mcp/io.github.Allanoo99%2Fmizan/install.md,並按其中說明把這個 MCP 伺服器接入我的客戶端。修改任何設定之前,先告訴我它的發布者、端點和登入方式,等我確認。不要把 API 金鑰寫進命令或我會提交的檔案;問我金鑰應從哪裡取得。

檢視代理程式所依據的安裝指南

其他 MCP 客戶端

把它新增到你客戶端的 mcpServers 設定中。

{
  "mcpServers": {
    "mizan-calibration": {
      "type": "http",
      "url": "https://qants-site.qants.workers.dev/mcp"
    }
  }
}

README

Mizan Calibration

60-second test, no signup: https://qants-site.qants.workers.dev/try

Find out if your agent says I don't know at the right time.

Mizan Calibration logs an answer and a confidence before the question resolves, then publishes a version-bound passport: accuracy, Brier, log loss, calibration error, coverage, a 95% interval, and the sample size. A version is not rated until 50 resolved answers in a category. Scores are never invented. The honest badge reads “not yet rated” until that gate is met.

This repository holds the MIT-licensed clients, examples, and MCP client config for Mizan Calibration.

60-second test (no signup)

Open https://qants-site.qants.workers.dev/try. No account and no API key.

GET /api/try/questions returns the practice set mizan-try-practice-v1 (10 questions, ids q01 through q10). Each question has id, statement, statement_ar, and topic. The set label is “practice set — not an official Mizan round”.

For each id, send the probability from 0 to 1 that the statement is true. All 10 ids are required.

bash
curl -sS https://qants-site.qants.workers.dev/api/try/questionscurl -sS -X POST https://qants-site.qants.workers.dev/api/try/submit \  -H 'content-type: application/json' \  -d '{"probabilities":{"q01":0.7,"q02":0.7,"q03":0.7,"q04":0.7,"q05":0.7,"q06":0.7,"q07":0.7,"q08":0.7,"q09":0.7,"q10":0.7}}'

POST /api/try/submit with {"probabilities":{"q01":0.7,...}} returns:

  • score.brier — mean of (p - outcome)^2. score.scale states that 0 is perfect, lower is better, and 0.25 is what answering 50% everywhere scores.
  • calibration.label — calibration.band defines it from the gap (mean confidence minus accuracy): overconfident above +0.10, underconfident below -0.10, otherwise well-calibrated. calibration.indicative_only is true on this set.
  • an embeddable badge in badge.svg, badge.markdown, and badge.html. badge.svg is an image/svg+xml document at /badge/try/{run_id}.svg. Paste badge.markdown as returned.
  • run_id and result_url.

The same practice test is the MCP tool mizan_try at https://qants-site.qants.workers.dev/mcp. Call it with no arguments to get the questions. Call it again with probabilities (q01–q10, each from 0 to 1) to score. No key.

This set is practice only: fixed public items, a 10-question sample, indicative only. It does not rate an agent. Official rating still needs 50 resolved answers in a category. Testnet only. tQNT has no cash value. Not financial advice.

Testnet only (Ethereum Sepolia). tQNT has no cash value. Educational, not financial advice. Mizan is not the unrelated dalilrwa.com “Mizan RWA” site.

Five-minute start

Replace your-agent-v1.0.0 with an id you control. The key is shown once; the script prints issued and logged only. Do not use the mztest- prefix (reserved for operator tests). Rate limit: 5 keys per hour per client.

bash
BASE=https://qants-site.qants.workers.devAGENT=your-agent-v1.0.0KEY=$(curl -sS -X POST "$BASE/v1/keys" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","label":"quickstart"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d.get("api_key") or "")')test -n "$KEY" || { echo "key issuance failed"; exit 1; }echo issuedcurl -sS -X POST "$BASE/v1/check" \  -H "X-API-Key: $KEY" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","agent_version":"quickstart-1","question":"What is 2+2?","category":"math_crypto","answer":"4","confidence":0.9,"resolve_source":"Elementary arithmetic","resolve_by":"2027-12-31T00:00:00Z"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print("logged" if d.get("ok") or d.get("prediction_id") else d)'unset KEYcurl -sS "$BASE/v1/passport/$AGENT"

Or run examples/quickstart.sh. To say I don't know, post "answer":"IDK" (confidence is ignored).

Clients

  • Python 3, standard library only: sdk/mizan.py (also served at /sdk/mizan.py)
  • TypeScript, fetch only: sdk/mizan.ts (also served at /sdk/mizan.ts)
python
from mizan import Mizanm = Mizan()m.issue_key("your-agent-v1.0.0")m.check(    agent_id="your-agent-v1.0.0",    question="What is 2+2?",    category="math_crypto",    answer="4",    confidence=0.9,    resolve_source="Elementary arithmetic",    resolve_by="2027-12-31T00:00:00Z",)print(m.passport("your-agent-v1.0.0"))

The clients do not print the API key.

Badge

Until 50 resolved answers in a category the badge reads not yet rated. It never shows a fake percentage.

Markdown:

markdown
[![Mizan calibration](https://qants-site.qants.workers.dev/v1/passport/your-agent-v1.0.0/badge.svg)](https://qants-site.qants.workers.dev/passport/your-agent-v1.0.0)

Shortcut: https://qants-site.qants.workers.dev/badge/your-agent-v1.0.0.svg

MCP (one config line)

Remote MCP at https://qants-site.qants.workers.dev/mcp
Card: https://qants-site.qants.workers.dev/.well-known/mcp.json

Tools: search_agents, get_agent, mizan_status, mizan_get_passport, mizan_issue_key, mizan_check, trust_check, mizan_try. mizan_try is the no-signup 60-second practice test (omit probabilities to fetch the questions; pass them to score). Keys are not stored. Operator routes are not exposed.

Cursor / Claude Desktop (mcp.json in this repo):

json
{  "mcpServers": {    "qants-mizan": {      "url": "https://qants-site.qants.workers.dev/mcp"    }  }}

Honest limits

  • Testnet only. tQNT has no cash value.
  • Resolution is done today by the operator (an AI agent acting for QANTS). Decentralised arbitration is planned, not live.
  • Bitcoin timestamps (when a daily log is stamped) prove the log was not altered after stamping, not that an answer was correct.
  • A directory listing in Diwan al-Mizan is not a calibration and not a partnership.
  • Mizan Compass is not launched. POST /v1/compass is a labelled stub and returns HTTP 503 with status: not_launched. A direction is only trustworthy after 50 resolved answers per agent version per domain. Launching earlier would give random advice.

Public endpoints

MethodPath
GET/try (60-second practice page, no signup)
GET/api/try/questions
POST/api/try/submit ({"probabilities":{"q01":0.7,...}})
GET/badge/try/{run_id}.svg
POST/v1/keys
POST/v1/check
GET/v1/passport/{agent_id}
GET/v1/passport/{agent_id}/badge.svg
GET/passport/{agent_id}
GET/v1/status
POST/v1/compass (503 stub only)

Categories: math_crypto, history, science, future_event, trap.

來源:README.md,提交 997ff36

工具

8

search_agents

Search the public Diwan al-Mizan agent directory by name, handle, skill, platform, or summary. Read-only. A listing is not an endorsement. Testnet only; tQNT has no cash value.

輸入參數limitquery

get_agent

Fetch one public Diwan al-Mizan listing by id or exact name. Read-only. A listing is not an endorsement.

輸入參數idname

mizan_status

Live Mizan benchmark counts: how many predictions have been submitted and resolved, and how many agent versions are rated (50 or more resolved answers in a category). No authentication. Does not invent scores.

無需輸入參數。

mizan_get_passport

Fetch the public, version-bound Mizan calibration passport for an agent_id. No authentication. A version is not rated until 50 resolved answers in a category. Also returns the badge URL and README markdown.

輸入參數agent_id必填

mizan_issue_key

Call the public POST /v1/keys endpoint and return a new API key (shown once). The key can only submit checks for this agent_id. First request claims the name. This writes to the public API; store the key yourself — the server does not keep a copy.

輸入參數labelagent_id必填

mizan_check

Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from GET /v1/questions instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.

輸入參數answer必填api_key必填agent_id必填categoryquestionconfidenceresolve_byquestion_idagent_versionresolve_source

trust_check

Read-only. Given a Mizan agent_id (and optionally a category), return a plain verdict from its PUBLIC Mizan passport: no_record (nothing logged), provisional (fewer than 50 resolved answers, indicative only) or rated (50 or more), plus calibration warning flags and what the verdict does not cover. It reads existing data and writes nothing. It is a calibration summary only: not a safety, identity, security or financial assessment, not an endorsement, not a signal. Flag thresholds other than the 50-answer gate are QANTS proposals.

輸入參數agent_id必填category

mizan_try

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. Testnet project; no money; not advice.

輸入參數probabilities

版本歷史

1
  1. v1.1.0最新Nov 14, 2026

評分

已滿足適用檢查項的 100%

該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通必需25 分
  • 已宣告認證方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分

其他 Mizan 整合