ML Referee

io.github.scarrico由 GitHub 账号 scarrico 发布v1.1.0更新于 Oct 29, 2026

Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta.

已验证Streamable HTTP可网页运行数据与分析AI 与机器学习

概览

AI 生成的概览

ML Referee 的远程 MCP 服务器，让助手上传去标识化数据集、基于基线报价并执行特征审计，并测试有界特征修复。

可以做什么
  • 了解服务能力、隐私条款、支持的评估配置与就绪状态
  • 上传加密的去标识化数据集、活动分片或修复矩阵，并在不消耗额度的情况下预检
  • 为审计、修复和活动报价，接受报价后提交并跟踪审计状态
  • 读取租户私有的完成结果、签名输出清单与报告回执，并校验报告哈希
  • 记录审计后续情况，删除保留的上传或审计
  • 查询审计额度余额、兑换测试邀请或申请测试访问
适用场景
适合希望让助手判断 ML 特征是否优于基线，或在私有数据上诊断效果下降并测试有界特征修复的场景，报价与额度管理也在同一连接内完成。该服务为邀请制测试版，适合已获授权或正在申请访问的团队。
运行要求
远程 streamable HTTP 端点 api.mlreferee.com；未声明本地包或环境变量。工具涉及已认证租户、凭据和审计额度，因此需要账号与测试访问权限。数据集以去标识化的 CSV 或 JSON 载荷上传。
安装前请注意
上传会把数据发送给第三方，尽管描述为加密且去标识化，仍需确认哪些数据可以离开本地环境。审计会消耗付费审计额度，提交审计即预留额度。相关工具可删除保留的上传和审计结果。测试邀请由运营方发放，并于 2027 年 1 月 1 日到期。

安装

在 SourceWeft 中

  1. 打开 控制台中的 ML Referee，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable，通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体，或 SourceWeft 对话中。智能体会读取该服务器的安装指南，先向你展示发布者、端点和登录方式，经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/io.github.scarrico%2Fmlreferee/install.md，并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前，先告诉我它的发布者、端点和登录方式，等我确认。不要把 API 密钥写进命令或我会提交的文件；问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "ml-referee": {
      "type": "http",
      "url": "https://api.mlreferee.com/mcp"
    }
  }
}

工具

26

describe_referee

Describe Referee

Discover capabilities, privacy, and the approved public catalog.

无需输入参数。

list_verticals

List evaluation profiles

List the three supported evaluation profiles and their discovery tags.

无需输入参数。

get_referee_readiness

Check Referee readiness

Report connection, credential, audit-credit, and submission readiness with the next action.

无需输入参数。

get_referee_usage

Read aggregate usage

Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.

输入参数window_days

get_input_schema

Get feature-audit schema

Get neutral column roles and validation defaults for a vertical.

输入参数vertical必填

get_repair_schema

Get feature-repair schema

Get the feature-only contract for diagnosing degradation and testing bounded repairs.

无需输入参数。

upload_private_dataset

Upload private dataset

Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.

输入参数data必填formatretention_seconds

upload_campaign_shard

Upload campaign shard

Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.

输入参数data必填formatretention_seconds

upload_repair_matrix

Upload repair matrix

Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.

输入参数data必填formatretention_seconds

preflight_dataset

Preflight feature audit

Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.

输入参数contract必填vertical必填upload_id必填

preflight_repair

Preflight feature repair

Validate a tenant-private feature-repair matrix without spending a credit.

输入参数contract必填upload_id必填

quote_audit

Quote feature audit

Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.

输入参数contract必填vertical必填upload_id必填idempotency_key必填

quote_repair

Quote feature repair

Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.

输入参数contract必填upload_id必填idempotency_key必填

quote_campaign

Quote feature campaign

Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.

输入参数shards必填vertical必填idempotency_key必填

get_credits

Read audit credits

Read beta or paid audit-credit availability and usage.

无需输入参数。

redeem_beta_invite

Redeem beta invite

Redeem one operator-issued invite for free beta audit credits before January 1, 2027.

输入参数invite_code必填

request_beta_access

Request beta access

Request operator-reviewed beta access without purchasing or granting credits.

输入参数lane必填use_case必填idempotency_key必填

get_beta_request_status

Get beta request status

Read the authenticated tenant's beta-access request status.

输入参数request_id必填

submit_audit

Submit audit

Accept a quote and atomically reserve its disclosed audit credits.

输入参数quote_id必填idempotency_key必填

record_audit_followup

Record audit follow-up

Record whether a completed audit changed a decision or was confirmed prospectively.

输入参数kind必填value必填audit_id必填

get_audit_status

Get audit status

Read the state of an audit owned by the authenticated tenant.

输入参数audit_id必填

get_audit_result

Get audit result

Read a completed tenant-private result and signed output manifest.

输入参数audit_id必填

get_audit_receipt

Get report receipt

Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.

输入参数audit_id必填

verify_audit_report

Verify report fingerprint

Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.

输入参数sha256必填audit_id必填representation

delete_audit

Delete audit

Delete retained inputs and results for an audit owned by this tenant.

输入参数audit_id必填

delete_upload

Delete upload

Delete a tenant-private upload that is no longer needed.

输入参数upload_id必填

版本历史

1
  1. v1.1.0最新Oct 29, 2026

评分

已满足适用检查项的 83%

该评分衡量此收录条目的完整程度，依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价，也不表示服务器安全或当前可用：可连通性是带日期的检查结果，评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分

    请声明包含 README 的 GitHub 仓库，或在 OAuth 受保护资源元数据中提供 resource_documentation。

  • 许可证不适用

    托管服务按其提供方的条款使用，因此这里不要求许可证。此检查项不计入百分比。