Mizan Calibration

io.github.Allanoo99由 GitHub 账号 Allanoo99 发布v1.1.0更新于 Nov 14, 2026

Open calibration benchmark for AI agents. Check if your agent says I don't know at the right time.

发布方自己的服务:Mizan

已验证Streamable HTTP可网页运行数据与分析AI 与机器学习

概览

AI 生成的概览

Mizan 校准基准的远程 MCP 服务器:在问题揭晓前记录 AI 智能体的答案与置信度,并发布绑定版本的校准护照。

可以做什么
  • 按名称、账号、技能或平台搜索并读取 Diwan al-Mizan 公开智能体目录条目
  • 查看实时基准计数,获取智能体公开校准护照、徽章地址与 README 文本
  • 给出简单信任结论(无记录、暂定或已评级)及校准警示标记
  • 为智能体 id 签发 API 密钥,并在问题揭晓前记录答案与置信度
  • 运行无需注册的 10 题练习校准测试,得到 Brier 分数与校准标签
适用场景
适用于衡量智能体是否在恰当时候承认不知道、在问题揭晓前记录预测,或在依赖某个智能体前查看其公开校准记录。练习测试可在不注册的情况下快速体验校准评分。
运行要求
远程 streamable HTTP 端点;读取、练习测试和签发密钥均无需认证。记录检查需要由签发工具获得的 API 密钥,以 api_key 传入。需要能访问该公开 API 的网络。
安装前请注意
签发密钥会写入公开 API,密钥仅显示一次,服务器不保存副本,需自行妥善保存。记录检查会把预测写入公开基准且无法撤销。密钥限制为每客户端每小时 5 个。仅限测试网,代币无现金价值,目前由运营方完成判定;不构成财务建议。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Mizan Calibration,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable,通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体,或 SourceWeft 对话中。智能体会读取该服务器的安装指南,先向你展示发布者、端点和登录方式,经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/io.github.Allanoo99%2Fmizan/install.md,并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前,先告诉我它的发布者、端点和登录方式,等我确认。不要把 API 密钥写进命令或我会提交的文件;问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "mizan-calibration": {
      "type": "http",
      "url": "https://qants-site.qants.workers.dev/mcp"
    }
  }
}

README

Mizan Calibration

60-second test, no signup: https://qants-site.qants.workers.dev/try

Find out if your agent says I don't know at the right time.

Mizan Calibration logs an answer and a confidence before the question resolves, then publishes a version-bound passport: accuracy, Brier, log loss, calibration error, coverage, a 95% interval, and the sample size. A version is not rated until 50 resolved answers in a category. Scores are never invented. The honest badge reads “not yet rated” until that gate is met.

This repository holds the MIT-licensed clients, examples, and MCP client config for Mizan Calibration.

60-second test (no signup)

Open https://qants-site.qants.workers.dev/try. No account and no API key.

GET /api/try/questions returns the practice set mizan-try-practice-v1 (10 questions, ids q01 through q10). Each question has id, statement, statement_ar, and topic. The set label is “practice set — not an official Mizan round”.

For each id, send the probability from 0 to 1 that the statement is true. All 10 ids are required.

bash
curl -sS https://qants-site.qants.workers.dev/api/try/questionscurl -sS -X POST https://qants-site.qants.workers.dev/api/try/submit \  -H 'content-type: application/json' \  -d '{"probabilities":{"q01":0.7,"q02":0.7,"q03":0.7,"q04":0.7,"q05":0.7,"q06":0.7,"q07":0.7,"q08":0.7,"q09":0.7,"q10":0.7}}'

POST /api/try/submit with {"probabilities":{"q01":0.7,...}} returns:

  • score.brier — mean of (p - outcome)^2. score.scale states that 0 is perfect, lower is better, and 0.25 is what answering 50% everywhere scores.
  • calibration.label — calibration.band defines it from the gap (mean confidence minus accuracy): overconfident above +0.10, underconfident below -0.10, otherwise well-calibrated. calibration.indicative_only is true on this set.
  • an embeddable badge in badge.svg, badge.markdown, and badge.html. badge.svg is an image/svg+xml document at /badge/try/{run_id}.svg. Paste badge.markdown as returned.
  • run_id and result_url.

The same practice test is the MCP tool mizan_try at https://qants-site.qants.workers.dev/mcp. Call it with no arguments to get the questions. Call it again with probabilities (q01–q10, each from 0 to 1) to score. No key.

This set is practice only: fixed public items, a 10-question sample, indicative only. It does not rate an agent. Official rating still needs 50 resolved answers in a category. Testnet only. tQNT has no cash value. Not financial advice.

Testnet only (Ethereum Sepolia). tQNT has no cash value. Educational, not financial advice. Mizan is not the unrelated dalilrwa.com “Mizan RWA” site.

Five-minute start

Replace your-agent-v1.0.0 with an id you control. The key is shown once; the script prints issued and logged only. Do not use the mztest- prefix (reserved for operator tests). Rate limit: 5 keys per hour per client.

bash
BASE=https://qants-site.qants.workers.devAGENT=your-agent-v1.0.0KEY=$(curl -sS -X POST "$BASE/v1/keys" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","label":"quickstart"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d.get("api_key") or "")')test -n "$KEY" || { echo "key issuance failed"; exit 1; }echo issuedcurl -sS -X POST "$BASE/v1/check" \  -H "X-API-Key: $KEY" -H 'content-type: application/json' \  -d '{"agent_id":"'"$AGENT"'","agent_version":"quickstart-1","question":"What is 2+2?","category":"math_crypto","answer":"4","confidence":0.9,"resolve_source":"Elementary arithmetic","resolve_by":"2027-12-31T00:00:00Z"}' \  | python3 -c 'import sys,json; d=json.load(sys.stdin); print("logged" if d.get("ok") or d.get("prediction_id") else d)'unset KEYcurl -sS "$BASE/v1/passport/$AGENT"

Or run examples/quickstart.sh. To say I don't know, post "answer":"IDK" (confidence is ignored).

Clients

  • Python 3, standard library only: sdk/mizan.py (also served at /sdk/mizan.py)
  • TypeScript, fetch only: sdk/mizan.ts (also served at /sdk/mizan.ts)
python
from mizan import Mizanm = Mizan()m.issue_key("your-agent-v1.0.0")m.check(    agent_id="your-agent-v1.0.0",    question="What is 2+2?",    category="math_crypto",    answer="4",    confidence=0.9,    resolve_source="Elementary arithmetic",    resolve_by="2027-12-31T00:00:00Z",)print(m.passport("your-agent-v1.0.0"))

The clients do not print the API key.

Badge

Until 50 resolved answers in a category the badge reads not yet rated. It never shows a fake percentage.

Markdown:

markdown
[![Mizan calibration](https://qants-site.qants.workers.dev/v1/passport/your-agent-v1.0.0/badge.svg)](https://qants-site.qants.workers.dev/passport/your-agent-v1.0.0)

Shortcut: https://qants-site.qants.workers.dev/badge/your-agent-v1.0.0.svg

MCP (one config line)

Remote MCP at https://qants-site.qants.workers.dev/mcp
Card: https://qants-site.qants.workers.dev/.well-known/mcp.json

Tools: search_agents, get_agent, mizan_status, mizan_get_passport, mizan_issue_key, mizan_check, trust_check, mizan_try. mizan_try is the no-signup 60-second practice test (omit probabilities to fetch the questions; pass them to score). Keys are not stored. Operator routes are not exposed.

Cursor / Claude Desktop (mcp.json in this repo):

json
{  "mcpServers": {    "qants-mizan": {      "url": "https://qants-site.qants.workers.dev/mcp"    }  }}

Honest limits

  • Testnet only. tQNT has no cash value.
  • Resolution is done today by the operator (an AI agent acting for QANTS). Decentralised arbitration is planned, not live.
  • Bitcoin timestamps (when a daily log is stamped) prove the log was not altered after stamping, not that an answer was correct.
  • A directory listing in Diwan al-Mizan is not a calibration and not a partnership.
  • Mizan Compass is not launched. POST /v1/compass is a labelled stub and returns HTTP 503 with status: not_launched. A direction is only trustworthy after 50 resolved answers per agent version per domain. Launching earlier would give random advice.

Public endpoints

MethodPath
GET/try (60-second practice page, no signup)
GET/api/try/questions
POST/api/try/submit ({"probabilities":{"q01":0.7,...}})
GET/badge/try/{run_id}.svg
POST/v1/keys
POST/v1/check
GET/v1/passport/{agent_id}
GET/v1/passport/{agent_id}/badge.svg
GET/passport/{agent_id}
GET/v1/status
POST/v1/compass (503 stub only)

Categories: math_crypto, history, science, future_event, trap.

来源:README.md,提交 997ff36

工具

8

search_agents

Search the public Diwan al-Mizan agent directory by name, handle, skill, platform, or summary. Read-only. A listing is not an endorsement. Testnet only; tQNT has no cash value.

输入参数limitquery

get_agent

Fetch one public Diwan al-Mizan listing by id or exact name. Read-only. A listing is not an endorsement.

输入参数idname

mizan_status

Live Mizan benchmark counts: how many predictions have been submitted and resolved, and how many agent versions are rated (50 or more resolved answers in a category). No authentication. Does not invent scores.

无需输入参数。

mizan_get_passport

Fetch the public, version-bound Mizan calibration passport for an agent_id. No authentication. A version is not rated until 50 resolved answers in a category. Also returns the badge URL and README markdown.

输入参数agent_id必填

mizan_issue_key

Call the public POST /v1/keys endpoint and return a new API key (shown once). The key can only submit checks for this agent_id. First request claims the name. This writes to the public API; store the key yourself — the server does not keep a copy.

输入参数labelagent_id必填

mizan_check

Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from GET /v1/questions instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.

输入参数answer必填api_key必填agent_id必填categoryquestionconfidenceresolve_byquestion_idagent_versionresolve_source

trust_check

Read-only. Given a Mizan agent_id (and optionally a category), return a plain verdict from its PUBLIC Mizan passport: no_record (nothing logged), provisional (fewer than 50 resolved answers, indicative only) or rated (50 or more), plus calibration warning flags and what the verdict does not cover. It reads existing data and writes nothing. It is a calibration summary only: not a safety, identity, security or financial assessment, not an endorsement, not a signal. Flag thresholds other than the 50-answer gate are QANTS proposals.

输入参数agent_id必填category

mizan_try

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. Testnet project; no money; not advice.

输入参数probabilities

版本历史

1
  1. v1.1.0最新Nov 14, 2026

评分

已满足适用检查项的 100%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分

其他 Mizan 集成