Web Search API Eval

io.github.psinglap由 GitHub 帳號 psinglap 發布v1.1.0更新於 Oct 29, 2026

Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.

已驗證Streamable HTTP可網頁執行網頁搜尋與抓取資料與分析

概覽

AI 產生的概覽

提供一份已發布的網路搜尋 API 基準評測（You.com、Tavily、Exa、Parallel），協助助理判斷哪個 API 適合某類代理需求。

可以做什麼
  • 讀取總體結論、整體與分場景評分卡，以及可依指標篩選的供應商比較
  • 取得針對特定代理需求的搜尋 API 建議與佐證數據
  • 瀏覽 30 道基準問題、查看單一題目的答案要點與各 API 分數，並檢視已發布圖表
  • 閱讀評測方法與在本機重跑基準的步驟
  • 依 ID 搜尋並取得基準文件全文
  • 使用自己的 API 金鑰對各供應商做一次即時抽查（僅限本機安裝）
適用情境
適合在挑選或說明 AI 代理的網路搜尋 API 時使用，例如比較延遲、準確率、時效性或成本，或查詢某道基準問題的評分方式。資料是固定快照，並非即時搜尋工具。
執行需求
可使用託管的遠端端點（免安裝、免驗證），或以 npx 執行本機 npm 套件，需要 Node.js 18 以上版本。選用金鑰變數 EXA_API_KEY、PARALLEL_API_KEY、TAVILY_API_KEY、YDC_API_KEY 僅供本機即時搜尋工具使用。
安裝前請注意
基準工具為唯讀，提供的是某次執行日期的固定快照。本機即時搜尋工具會用你的金鑰向各供應商送出查詢並消耗你自己的額度，且僅為一次抽查，不是正式評分基準。金鑰皆為選用，沒有金鑰的供應商可略過。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 Web Search API Eval，將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Web executable，透過 Streamable HTTP。 遠端服務在工作區中設定後即可從網頁執行環境執行。

交給代理程式安裝

把這段提示詞貼上到 Claude Code、Codex、Cursor 等代理程式，或 SourceWeft 對話中。代理程式會讀取該伺服器的安裝指南，先向你展示發布者、端點和登入方式，經你同意後再把它新增到你的客戶端。金鑰不會寫進任何命令。

閱讀 https://sourceweft.com/mcp/io.github.psinglap%2Fwebsearch-api-eval/install.md，並按其中說明把這個 MCP 伺服器接入我的客戶端。修改任何設定之前，先告訴我它的發布者、端點和登入方式，等我確認。不要把 API 金鑰寫進命令或我會提交的檔案；問我金鑰應從哪裡取得。

檢視代理程式所依據的安裝指南

其他 MCP 客戶端

把它新增到你客戶端的 mcpServers 設定中。

{
  "mcpServers": {
    "web-search-api-eval": {
      "type": "http",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

README

Web Search API Eval: MCP server

An MCP server that lets Claude, ChatGPT, Cursor and other AI assistants answer questions about the Web Search API Eval benchmark (You.com, Tavily, Exa and Parallel on 30 agent queries), and run your own live search spot checks.

Use it two ways:

Hosted (no install)Local (npm)
Connecthttps://websearch-api-eval.psberkeleymba.workers.dev/mcpnpx -y websearch-api-eval-mcp
Works inClaude, ChatGPT, any remote MCP clientClaude Desktop, Claude Code, Cursor, VS Code, any stdio MCP client
Benchmark tools✅✅
run_live_search with your own keys—✅

Install

Claude Code

bash
claude mcp add websearch-api-eval -- npx -y websearch-api-eval-mcp

Claude Desktop, Cursor, VS Code (add to the client's MCP config):

json
{  "mcpServers": {    "websearch-api-eval": {      "command": "npx",      "args": ["-y", "websearch-api-eval-mcp"],      "env": {        "YDC_API_KEY": "optional",        "TAVILY_API_KEY": "optional",        "EXA_API_KEY": "optional",        "PARALLEL_API_KEY": "optional"      }    }  }}

The keys are optional and only used by run_live_search. Leave out any provider you don't have a key for. Requires Node.js 18 or later.

Hosted, no install: in Claude, Settings → Connectors → Add custom connector → paste the hosted URL. In ChatGPT, create an app in Developer mode with the same URL and no authentication.

Tools

ToolReturns
get_overall_resultsHeadline, overall scorecard, per-use-case summary, which API fits which agent, developer activation
get_scorecardFull scorecard for overall, gtm, insurance or finance
compare_providersTwo or more providers side by side, optionally filtered by metric keyword
recommend_providerWhich API fits an agent's need, with supporting numbers
list_questionsThe 30 questions, filterable by use case
get_questionOne question: task, search call, answer key, sources, what each API returned and its score
get_chartsThe published graphs; with chart_id, also the image
get_methodologyHow tests ran and were graded, the 10 metrics and their standards, limits
how_to_run_locallySteps and keys to rerun the full benchmark on your own machine
search, fetchGeneric search and document fetch (used by ChatGPT research features)
run_live_search (local only)Sends one query to each provider using your own API keys and returns latency, result count and top results

The benchmark tools are read-only and read the published results (run Oct 8, 2026). run_live_search calls the providers directly from your machine with your keys and uses your own credits; a single call is a spot check, not a graded benchmark. For the full graded run, use the harness.

Example questions

  • "Which web search API is fastest for a voice agent?"
  • "Compare Exa and Parallel on finance."
  • "What was question F5 and who got it right?"
  • "Show me the overall scorecard chart."
  • "Run a live search for 'FOMC minutes' and compare the providers."

Data and updates

Results come from data/*.json on the published site, built from the Evaluation page by data/build_data.py. After a new benchmark run, rebuild the data and push; both the hosted and npm servers pick it up with no redeploy or new release.

Develop

bash
cd mcpnpm installnpx wrangler dev        # hosted server locally at http://localhost:8787/mcpnpm run build           # npm package entry at dist/cli.jsnpx @modelcontextprotocol/[email protected] node dist/cli.js   # call tools by handnpx wrangler deploy     # deploy the hosted servernpm publish             # publish the npm package

Built with the official MCP TypeScript SDK. Registry entry: io.github.psinglap/websearch-api-eval (server.json).

Privacy and support

No sign-in, no cookies, no storage, no request logging. Privacy Policy · Terms · Support · MIT License

來源：mcp/README.md，提交 513d05b

工具

11

get_overall_results

Overall results

Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.

無需輸入參數。

get_scorecard

Scorecard

Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.

輸入參數use_case必填

compare_providers

Compare providers

Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').

輸入參數use_caseproviders必填metric_filter

recommend_provider

Recommend a provider

Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.

輸入參數agent_need

list_questions

List questions

The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.

輸入參數use_case

get_question

Get a question

One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).

輸入參數id必填

get_charts

Charts

The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.

輸入參數chart_iduse_case

get_methodology

Methodology

How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.

無需輸入參數。

how_to_run_locally

Run it yourself

How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.

無需輸入參數。

fetch

Fetch a benchmark document

Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).

輸入參數id必填

版本歷史

1
  1. v1.1.0最新Oct 29, 2026

評分

已滿足適用檢查項的 100%

該評分衡量此收錄條目的完整程度，依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價，也不表示伺服器安全或目前可用：可連通性是帶日期的檢查結果，評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通必需25 分
  • 已宣告認證方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分