AgentTrust

io.github.vitaliiserbynassisterr由 GitHub 帳號 vitaliiserbynassisterr 發布v0.1.0更新於 Oct 24, 2026

Quality verification for AI agents and MCP servers. 6-axis scoring, adversarial probes.

已驗證STDIO僅桌面安全與監控AI 與機器學習

概覽

AI 產生的概覽

AgentTrust 用來評估 AI 代理與 MCP 伺服器的品質與安全性，依六個面向評分並簽發可驗證憑證。

可以做什麼
對 MCP 伺服器網址執行完整或快速品質檢查，涵蓋清單、功能工具呼叫與 LLM 評審評分，包含準確性、安全性、可靠性等六個面向，並含提示注入、PII 外洩等對抗性探測。查詢具時效衰減的快取品質分數，並驗證簽章憑證權杖。在盲測對戰競技場中兩兩比較代理，提供貝氏評分與排行榜，並透過 IRT 校準自適應挑選測試題目。
適用情境
當你想在把實際任務或付款交給某個 AI 代理或 MCP 伺服器之前先做品質審查，或需要為正在整合或比較的代理取得可驗證的評分訊號時使用。
執行需求
以本機 Python 程序執行（PyPI 套件 mcp-agenttrust，stdio）或透過 Docker 執行；需要至少一個 LLM 服務商金鑰（CEREBRAS_API_KEY 或 GROQ_API_KEY，其餘服務商構成備援鏈）以及用於持久化儲存的 MongoDB 連線位址（MONGODB_URL）；完整部署另涉及 Redis 與 Solana 錢包。
安裝前請注意
需要機密憑證：CEREBRAS_API_KEY、GROQ_API_KEY 與 MONGODB_URL。評估會把目標伺服器資訊與測試提示傳送給第三方 LLM 服務商。完整部署包含 x402 Solana 付款驗證與 Solana 錢包設定，使用前應檢視相關付款設定。評估結果會存入 MongoDB。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 AgentTrust，將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Desktop only，透過 STDIO。 STDIO 服務會啟動本機處理程序，因此需要 SourceWeft 桌面主機。

其他 MCP 客戶端

參照 儲存庫 中的啟動說明。

README

AgentTrust

Challenge-response quality verification for AI agents and MCP servers.

AgentTrust evaluates AI agent competency before you trust them with real tasks or payments. It connects to any MCP server, runs challenge-response tests across 6 quality dimensions, and issues W3C Verifiable Credentials as proof.

Why

The AI agent ecosystem has identity (ERC-8004, SATI), post-hoc reputation (TARS, Amiko), and payments (x402) — but no pre-payment quality gate. AgentTrust fills this gap: verify competency first, then trust.

Features

Evaluation Engine

  • 3-level pipeline: Manifest (schema) → Functional (tool calls) → Domain Expert (calibrated questions)
  • 6-axis scoring: accuracy (35%), safety (20%), reliability (15%), process quality (10%), latency (10%), schema quality (10%)
  • Consensus judging: 2-3 LLM judges in parallel with agreement threshold (saves 50-66% LLM calls)
  • 7 LLM provider fallback chain: Cerebras → Groq → OpenRouter → Gemini → Mistral → DeepSeek → OpenAI
  • 5 adversarial probe types: prompt injection, PII leakage, hallucination, overflow, system prompt extraction

Battle Arena

  • Head-to-head blind evaluation with position-swap consistency
  • OpenSkill (Bayesian ELO) rating system with divisions (Bronze → Grandmaster)
  • Fair matchmaking: rating proximity + uncertainty bonus + cross-division challenges
  • Style control penalties to prevent gaming via verbose/formatted responses

IRT Adaptive Testing

  • Rasch 1PL calibration from battle data (pure Python, no numpy)
  • Fisher information maximization for adaptive question selection
  • EAP ability estimation with standard normal prior
  • Reduces evaluation cost by 50-90% while maintaining accuracy

Standards

  • W3C Verifiable Credentials (AQVC format) with Ed25519 DataIntegrityProof
  • Google A2A v0.3 native support (AgentTrust IS an A2A agent)
  • x402 Solana payment verification (USDC + SOL)
  • AIUC-1 protocol mapping

Quick Start

Docker (recommended)

bash
cp .env.example .env# Add at least one LLM key (GROQ_API_KEY, CEREBRAS_API_KEY, etc.)docker compose up -d

Services:

Local Development

bash
python -m venv .venv && source .venv/bin/activatepip install -r requirements.txtcp .env.example .env# Add LLM keys to .env
unset GROQ_API_KEY  # Shell env overrides .env rotation poolpython -m uvicorn src.main:app --host 0.0.0.0 --port 8002 --reload

MCP Server (for Claude, Cursor, Windsurf)

Add to your MCP client config:

json
{  "mcpServers": {    "agenttrust": {      "command": "python",      "args": ["-m", "src.standards.mcp_server"],      "env": {        "GROQ_API_KEY": "your-key"      }    }  }}

Or connect to a running instance via SSE:

http://localhost:8003/sse

Available MCP tools:

ToolDescription
check_quality(server_url)Full evaluation: manifest + functional + judge scoring
check_quality_fast(server_url)Cached score (<10ms) or manifest-only (<100ms)
get_score(server_url)Lookup cached score with freshness decay
verify_attestation(attestation_jwt)Verify AQVC JWT and decode payload

API Endpoints

MethodEndpointDescription
POST/v1/evaluateSubmit target for evaluation
GET/v1/evaluate/{id}Poll evaluation status
GET/v1/score/{target_id}Get quality score
GET/v1/scoresSearch/list scores
GET/v1/badge/{target_id}.svgSVG quality badge
GET/v1/attestation/{id}Get signed attestation (JWT or W3C VC)
POST/v1/attestation/{id}/verifyVerify attestation
POST/v1/feedbackSubmit production feedback (anti-sandbagging)
POST/v1/battlesCreate evaluation battle
GET/v1/arena/leaderboardBattle arena leaderboard
GET/v1/rankingsGlobal rankings by domain/tier
POST/v1/irt/calibrateTrigger IRT batch calibration
GET/v1/irt/recommendAdaptive question selection
GET/v1/pricingx402 pricing table
GET/.well-known/agent.jsonA2A Agent Card

Architecture

src/  api/v1/          # 14 FastAPI routers  core/            # Evaluator, MCP client, scoring, IRT, battle arena  auth/            # API keys (SHA256 + salt), rate limiting by tier  storage/         # MongoDB (Motor) + Redis  payments/        # x402 protocol, Solana verification  standards/       # W3C VC issuer, A2A extension, MCP server, AIUC-1

Stack: FastAPI + MongoDB + Redis | 533 tests | 60 source files | 15 lean dependencies

Tests

bash
python -m pytest tests/ -q# 533 passed in ~2s

Configuration

See .env.example for all 60+ configuration options including:

  • LLM API keys (7 providers, comma-separated for rotation)
  • MongoDB/Redis connection
  • JWT attestation (Ed25519 key, issuer DID, validity)
  • Solana wallet for x402 payments
  • Rate limit tiers and consensus judge settings

License

MIT

Links

Built by Assisterr

來源：README.md，提交 6152a08

工具

0
SourceWeft 尚未在沙箱中啟動過該軟體包，因此其工具未知。

版本歷史

1
  1. v0.1.0最新Sep 16, 2026

評分

已滿足適用檢查項的 50%

該評分衡量此收錄條目的完整程度，依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價，也不表示伺服器安全或目前可用：可連通性是帶日期的檢查結果，評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通25 分

    請保持端點能夠回應 MCP 握手；SourceWeft 每 30 天探測一次。

  • 已宣告認證方式10 分

    請在註冊中心條目中宣告憑證請求標頭，或保持端點能夠回應探測，以便判定其認證方式。

  • 已列出工具15 分

    請讓伺服器在無憑證時回應 tools/list，或發布可在沙箱中無憑證啟動的軟體包。

  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分