io.github.psinglapPublished by GitHub account psinglapv1.1.0Updated Oct 29, 2026
Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
Serves a published benchmark comparing four web search APIs (You.com, Tavily, Exa, Parallel) so an assistant can answer which one fits an agent's needs.
Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.
Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.
Read https://sourceweft.com/mcp/io.github.psinglap%2Fwebsearch-api-eval/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.Add this to your client's mcpServers config.
{
"mcpServers": {
"web-search-api-eval": {
"type": "http",
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}An MCP server that lets Claude, ChatGPT, Cursor and other AI assistants answer questions about the Web Search API Eval benchmark (You.com, Tavily, Exa and Parallel on 30 agent queries), and run your own live search spot checks.
Use it two ways:
Claude Code
Claude Desktop, Cursor, VS Code (add to the client's MCP config):
The keys are optional and only used by run_live_search. Leave out any provider you don't have a key for. Requires Node.js 18 or later.
Hosted, no install: in Claude, Settings → Connectors → Add custom connector → paste the hosted URL. In ChatGPT, create an app in Developer mode with the same URL and no authentication.
The benchmark tools are read-only and read the published results (run Oct 8, 2026). run_live_search calls the providers directly from your machine with your keys and uses your own credits; a single call is a spot check, not a graded benchmark. For the full graded run, use the harness.
Results come from data/*.json on the published site, built from the Evaluation page by data/build_data.py. After a new benchmark run, rebuild the data and push; both the hosted and npm servers pick it up with no redeploy or new release.
Built with the official MCP TypeScript SDK. Registry entry: io.github.psinglap/websearch-api-eval (server.json).
No sign-in, no cookies, no storage, no request logging. Privacy Policy · Terms · Support · MIT License
Source: mcp/README.md at commit 513d05b
Overall results
Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.
Takes no parameters.
Scorecard
Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.
use_caserequiredCompare providers
Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').
use_caseprovidersrequiredmetric_filterRecommend a provider
Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.
agent_needList questions
The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.
use_caseGet a question
One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).
idrequiredCharts
The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.
chart_iduse_caseMethodology
How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.
Takes no parameters.
Run it yourself
How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.
Takes no parameters.
Search the benchmark
Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.
queryrequiredFetch a benchmark document
Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).
idrequired100% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
claude mcp add websearch-api-eval -- npx -y websearch-api-eval-mcp{ "mcpServers": { "websearch-api-eval": { "command": "npx", "args": ["-y", "websearch-api-eval-mcp"], "env": { "YDC_API_KEY": "optional", "TAVILY_API_KEY": "optional", "EXA_API_KEY": "optional", "PARALLEL_API_KEY": "optional" } } }}cd mcpnpm installnpx wrangler dev # hosted server locally at http://localhost:8787/mcpnpm run build # npm package entry at dist/cli.jsnpx @modelcontextprotocol/[email protected] node dist/cli.js # call tools by handnpx wrangler deploy # deploy the hosted servernpm publish # publish the npm package