AgentTrust

io.github.assister-xyz由 GitHub 账号 assister-xyz 发布v0.1.2更新于 Oct 24, 2026

Quality verification for AI agents and MCP servers. 6-axis scoring, adversarial probes.

已验证STDIO仅桌面安全与监控AI 与机器学习

概览

AI 生成的概览

AgentTrust 用于评估其他 AI 智能体和 MCP 服务器的质量,按六个维度打分并签发可验证凭证作为证明。

可以做什么
对目标 MCP 服务器执行完整质量评估:清单校验、功能工具调用与评审打分,覆盖六个加权维度,并包含提示注入、PII 泄露、幻觉等对抗性探测。获取快速缓存分数或仅基于清单的分数,或查询已有分数并考虑时效衰减。验证 AQVC 凭证令牌并解码其内容。工具背后还有对战竞技场排名、自适应 IRT 测试,以及用于评估、排行榜、徽章和凭证的 HTTP API。
适用场景
适合在把真实任务交给第三方智能体或 MCP 服务器之前做信任前检查,或需要为自己发布的服务器提供可验证的签名质量凭证时使用。
运行要求
以本地 Python 进程运行(PyPI 包 mcp-agenttrust,stdio),也可用 Docker 部署。至少需要一个 LLM 服务商密钥(CEREBRAS_API_KEY 或 GROQ_API_KEY,README 还列出更多备选)以及 MongoDB 连接地址(MONGODB_URL);完整部署还会用到 Redis。需要访问目标服务器和 LLM 服务商的网络。
安装前请注意
需要密钥凭据:CEREBRAS_API_KEY、GROQ_API_KEY 和 MONGODB_URL,另可选 Ed25519 凭证密钥和用于 x402 支付的 Solana 钱包。评估会把数据发送给第三方 LLM 服务商,并连接目标服务器,可能对其发起对抗性探测。README 提到 USDC/SOL 支付验证,使用前应审查相关支付配置。

安装

在 SourceWeft 中

  1. 打开 控制台中的 AgentTrust,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Desktop only,通过 STDIO。 STDIO 服务会启动本地进程,因此需要 SourceWeft 桌面宿主。

其他 MCP 客户端

参照 仓库 中的启动说明。

README

AgentTrust

Challenge-response quality verification for AI agents and MCP servers.

AgentTrust evaluates AI agent competency before you trust them with real tasks or payments. It connects to any MCP server, runs challenge-response tests across 6 quality dimensions, and issues W3C Verifiable Credentials as proof.

Why

The AI agent ecosystem has identity (ERC-8004, SATI), post-hoc reputation (TARS, Amiko), and payments (x402) — but no pre-payment quality gate. AgentTrust fills this gap: verify competency first, then trust.

Features

Evaluation Engine

  • 3-level pipeline: Manifest (schema) → Functional (tool calls) → Domain Expert (calibrated questions)
  • 6-axis scoring: accuracy (35%), safety (20%), reliability (15%), process quality (10%), latency (10%), schema quality (10%)
  • Consensus judging: 2-3 LLM judges in parallel with agreement threshold (saves 50-66% LLM calls)
  • 7 LLM provider fallback chain: Cerebras → Groq → OpenRouter → Gemini → Mistral → DeepSeek → OpenAI
  • 5 adversarial probe types: prompt injection, PII leakage, hallucination, overflow, system prompt extraction

Battle Arena

  • Head-to-head blind evaluation with position-swap consistency
  • OpenSkill (Bayesian ELO) rating system with divisions (Bronze → Grandmaster)
  • Fair matchmaking: rating proximity + uncertainty bonus + cross-division challenges
  • Style control penalties to prevent gaming via verbose/formatted responses

IRT Adaptive Testing

  • Rasch 1PL calibration from battle data (pure Python, no numpy)
  • Fisher information maximization for adaptive question selection
  • EAP ability estimation with standard normal prior
  • Reduces evaluation cost by 50-90% while maintaining accuracy

Standards

  • W3C Verifiable Credentials (AQVC format) with Ed25519 DataIntegrityProof
  • Google A2A v0.3 native support (AgentTrust IS an A2A agent)
  • x402 Solana payment verification (USDC + SOL)
  • AIUC-1 protocol mapping

Quick Start

Docker (recommended)

bash
cp .env.example .env# Add at least one LLM key (GROQ_API_KEY, CEREBRAS_API_KEY, etc.)docker compose up -d

Services:

Local Development

bash
python -m venv .venv && source .venv/bin/activatepip install -r requirements.txtcp .env.example .env# Add LLM keys to .env
unset GROQ_API_KEY  # Shell env overrides .env rotation poolpython -m uvicorn src.main:app --host 0.0.0.0 --port 8002 --reload

MCP Server (for Claude, Cursor, Windsurf)

Add to your MCP client config:

json
{  "mcpServers": {    "agenttrust": {      "command": "python",      "args": ["-m", "src.standards.mcp_server"],      "env": {        "GROQ_API_KEY": "your-key"      }    }  }}

Or connect to a running instance via SSE:

http://localhost:8003/sse

Available MCP tools:

ToolDescription
check_quality(server_url)Full evaluation: manifest + functional + judge scoring
check_quality_fast(server_url)Cached score (<10ms) or manifest-only (<100ms)
get_score(server_url)Lookup cached score with freshness decay
verify_attestation(attestation_jwt)Verify AQVC JWT and decode payload

API Endpoints

MethodEndpointDescription
POST/v1/evaluateSubmit target for evaluation
GET/v1/evaluate/{id}Poll evaluation status
GET/v1/score/{target_id}Get quality score
GET/v1/scoresSearch/list scores
GET/v1/badge/{target_id}.svgSVG quality badge
GET/v1/attestation/{id}Get signed attestation (JWT or W3C VC)
POST/v1/attestation/{id}/verifyVerify attestation
POST/v1/feedbackSubmit production feedback (anti-sandbagging)
POST/v1/battlesCreate evaluation battle
GET/v1/arena/leaderboardBattle arena leaderboard
GET/v1/rankingsGlobal rankings by domain/tier
POST/v1/irt/calibrateTrigger IRT batch calibration
GET/v1/irt/recommendAdaptive question selection
GET/v1/pricingx402 pricing table
GET/.well-known/agent.jsonA2A Agent Card

Architecture

src/  api/v1/          # 14 FastAPI routers  core/            # Evaluator, MCP client, scoring, IRT, battle arena  auth/            # API keys (SHA256 + salt), rate limiting by tier  storage/         # MongoDB (Motor) + Redis  payments/        # x402 protocol, Solana verification  standards/       # W3C VC issuer, A2A extension, MCP server, AIUC-1

Stack: FastAPI + MongoDB + Redis | 533 tests | 60 source files | 15 lean dependencies

Tests

bash
python -m pytest tests/ -q# 533 passed in ~2s

Configuration

See .env.example for all 60+ configuration options including:

  • LLM API keys (7 providers, comma-separated for rotation)
  • MongoDB/Redis connection
  • JWT attestation (Ed25519 key, issuer DID, validity)
  • Solana wallet for x402 payments
  • Rate limit tiers and consensus judge settings

License

MIT

Links

Built by Assisterr

来源:README.md,提交 6152a08

工具

0
SourceWeft 尚未在沙箱中启动过该软件包,因此其工具未知。

版本历史

1
  1. v0.1.2最新Sep 16, 2026

评分

已满足适用检查项的 50%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通25 分

    请保持端点能够响应 MCP 握手;SourceWeft 每 30 天探测一次。

  • 已声明认证方式10 分

    请在注册中心条目中声明凭据请求头,或保持端点能够响应探测,以便判定其认证方式。

  • 已列出工具15 分

    请让服务器在无凭据时响应 tools/list,或发布可在沙箱中无凭据启动的软件包。

  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分