Web Search API Eval

io.github.psinglap由 GitHub 账号 psinglap 发布v1.1.0更新于 Oct 29, 2026

Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.

已验证Streamable HTTP可网页运行网页搜索与抓取数据与分析

概览

AI 生成的概览

提供一份已发布的网络搜索 API 基准评测（You.com、Tavily、Exa、Parallel），帮助助手判断哪个 API 适合某类智能体需求。

可以做什么
  • 读取总体结论、整体与分场景评分卡，以及按指标筛选的供应商对比
  • 获取针对特定智能体需求的搜索 API 推荐及支撑数据
  • 浏览 30 道基准问题、查看单题的答案要点与各 API 得分，并查看已发布图表
  • 阅读评测方法与在本地重跑基准的步骤
  • 按 ID 搜索并获取基准文档全文
  • 使用自己的 API 密钥对各供应商做一次实时抽查（仅本地安装）
适用场景
适合在选型或论证 AI 智能体的网络搜索 API 时使用，例如比较延迟、准确率、时效性或成本，或查询某道基准问题的评分方式。数据是固定快照，不是实时搜索工具。
运行要求
可使用托管远程端点（无需安装、无需认证），或通过 npx 运行本地 npm 包，需要 Node.js 18 或更高版本。可选密钥变量 EXA_API_KEY、PARALLEL_API_KEY、TAVILY_API_KEY、YDC_API_KEY 仅由本地实时搜索工具使用。
安装前请注意
基准工具为只读，提供的是某一次运行日期的固定快照。本地实时搜索工具会用你的密钥向各供应商发送查询并消耗你自己的额度，且只是一次抽查，不是正式评分基准。密钥均为可选，没有的供应商可以不填。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Web Search API Eval，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable，通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体，或 SourceWeft 对话中。智能体会读取该服务器的安装指南，先向你展示发布者、端点和登录方式，经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/io.github.psinglap%2Fwebsearch-api-eval/install.md，并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前，先告诉我它的发布者、端点和登录方式，等我确认。不要把 API 密钥写进命令或我会提交的文件；问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "web-search-api-eval": {
      "type": "http",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

README

Web Search API Eval: MCP server

An MCP server that lets Claude, ChatGPT, Cursor and other AI assistants answer questions about the Web Search API Eval benchmark (You.com, Tavily, Exa and Parallel on 30 agent queries), and run your own live search spot checks.

Use it two ways:

Hosted (no install)Local (npm)
Connecthttps://websearch-api-eval.psberkeleymba.workers.dev/mcpnpx -y websearch-api-eval-mcp
Works inClaude, ChatGPT, any remote MCP clientClaude Desktop, Claude Code, Cursor, VS Code, any stdio MCP client
Benchmark tools✅✅
run_live_search with your own keys—✅

Install

Claude Code

bash
claude mcp add websearch-api-eval -- npx -y websearch-api-eval-mcp

Claude Desktop, Cursor, VS Code (add to the client's MCP config):

json
{  "mcpServers": {    "websearch-api-eval": {      "command": "npx",      "args": ["-y", "websearch-api-eval-mcp"],      "env": {        "YDC_API_KEY": "optional",        "TAVILY_API_KEY": "optional",        "EXA_API_KEY": "optional",        "PARALLEL_API_KEY": "optional"      }    }  }}

The keys are optional and only used by run_live_search. Leave out any provider you don't have a key for. Requires Node.js 18 or later.

Hosted, no install: in Claude, Settings → Connectors → Add custom connector → paste the hosted URL. In ChatGPT, create an app in Developer mode with the same URL and no authentication.

Tools

ToolReturns
get_overall_resultsHeadline, overall scorecard, per-use-case summary, which API fits which agent, developer activation
get_scorecardFull scorecard for overall, gtm, insurance or finance
compare_providersTwo or more providers side by side, optionally filtered by metric keyword
recommend_providerWhich API fits an agent's need, with supporting numbers
list_questionsThe 30 questions, filterable by use case
get_questionOne question: task, search call, answer key, sources, what each API returned and its score
get_chartsThe published graphs; with chart_id, also the image
get_methodologyHow tests ran and were graded, the 10 metrics and their standards, limits
how_to_run_locallySteps and keys to rerun the full benchmark on your own machine
search, fetchGeneric search and document fetch (used by ChatGPT research features)
run_live_search (local only)Sends one query to each provider using your own API keys and returns latency, result count and top results

The benchmark tools are read-only and read the published results (run Oct 8, 2026). run_live_search calls the providers directly from your machine with your keys and uses your own credits; a single call is a spot check, not a graded benchmark. For the full graded run, use the harness.

Example questions

  • "Which web search API is fastest for a voice agent?"
  • "Compare Exa and Parallel on finance."
  • "What was question F5 and who got it right?"
  • "Show me the overall scorecard chart."
  • "Run a live search for 'FOMC minutes' and compare the providers."

Data and updates

Results come from data/*.json on the published site, built from the Evaluation page by data/build_data.py. After a new benchmark run, rebuild the data and push; both the hosted and npm servers pick it up with no redeploy or new release.

Develop

bash
cd mcpnpm installnpx wrangler dev        # hosted server locally at http://localhost:8787/mcpnpm run build           # npm package entry at dist/cli.jsnpx @modelcontextprotocol/[email protected] node dist/cli.js   # call tools by handnpx wrangler deploy     # deploy the hosted servernpm publish             # publish the npm package

Built with the official MCP TypeScript SDK. Registry entry: io.github.psinglap/websearch-api-eval (server.json).

Privacy and support

No sign-in, no cookies, no storage, no request logging. Privacy Policy · Terms · Support · MIT License

来源：mcp/README.md，提交 513d05b

工具

11

get_overall_results

Overall results

Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.

无需输入参数。

get_scorecard

Scorecard

Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.

输入参数use_case必填

compare_providers

Compare providers

Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').

输入参数use_caseproviders必填metric_filter

recommend_provider

Recommend a provider

Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.

输入参数agent_need

list_questions

List questions

The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.

输入参数use_case

get_question

Get a question

One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).

输入参数id必填

get_charts

Charts

The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.

输入参数chart_iduse_case

get_methodology

Methodology

How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.

无需输入参数。

how_to_run_locally

Run it yourself

How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.

无需输入参数。

fetch

Fetch a benchmark document

Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).

输入参数id必填

版本历史

1
  1. v1.1.0最新Oct 29, 2026

评分

已满足适用检查项的 100%

该评分衡量此收录条目的完整程度，依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价，也不表示服务器安全或当前可用：可连通性是带日期的检查结果，评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分