Crawler IP Verifier - real Googlebot or fake

dev.workers.pathwren.www由 www.pathwren.workers.dev 发布官方发布者v1.4.0更新于 Oct 24, 2026

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

发布方自己的服务:Agent Discovery Doctor - llms.txt, agent card, mcp.json

已验证Streamable HTTP可网页运行网页搜索与抓取安全与监控

概览

AI 生成的概览

一个远程、无需密钥的 MCP 端点,用镜像的爬虫运营方 IP 前缀列表核验地址,以区分真假 Googlebot 类爬虫。

可以做什么
报告本主机镜像了哪些爬虫运营方 IP 段列表,以及各来源的前缀数量、地址规模、发布时间与新鲜度,并列出被多个运营方共同发布的前缀。核验单个地址或调用方自身地址是否落在任一镜像的运营方前缀列表中。一次批量核验最多 500 个地址,并逐个返回匹配的前缀。查询某个 CIDR 被哪些已发布前缀包含、内含或部分重叠。把运营方地址段导出为可直接粘贴的 ACL 配置,例如 nginx、Apache、HAProxy 或 Cloudflare 防火墙规则。按爬虫列出验证方式,包括运营方是否发布 IP 段或提供反向 DNS 说明。
适用场景
适用于判断自称搜索引擎爬虫的访问者是否真实、审计或构建爬虫流量的 IP 允许/拒绝列表,以及在编写防火墙规则前评估爬虫地址段规模并确认数据新鲜度。
运行要求
远程 streamable HTTP 端点;未声明软件包、API 密钥、账户或请求头。需要能访问提供方主机的网络。
安装前请注意
whoami 与核验工具会返回调用方自身地址,因此该端点能看到连接来源 IP。结论依赖所镜像的运营方列表及其新鲜度,应视为参考。导出的 ACL 配置在应用到生产防火墙前应先审阅。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Crawler IP Verifier - real Googlebot or fake,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable,通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体,或 SourceWeft 对话中。智能体会读取该服务器的安装指南,先向你展示发布者、端点和登录方式,经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/dev.workers.pathwren.www%2Fcrawler-ip-verifier/install.md,并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前,先告诉我它的发布者、端点和登录方式,等我确认。不要把 API 密钥写进命令或我会提交的文件;问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "crawler-ip-verifier-real-googlebot-or-fa": {
      "type": "http",
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/netcheck"
    }
  }
}

工具

9

no_arguments_report_the_crawler_ip_ranges_this_host_mirrors

No arguments: every operator prefix list this host mirrors, and how stale each one is

TAKES NO ARGUMENTS. POST {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_report_the_crawler_ip_ranges_this_host_mirrors","arguments":{}}} to https://www.pathwren.workers.dev/mcp/netcheck — the answer is the state of every crawler-operator prefix list this host mirrors — how many IPv4 and IPv6 prefixes each source publishes and how many addresses that is, when each was last fetched and how many minutes ago that was, which crawlers each source covers, which prefixes more than one operator claims, and the verification method each operator documents for the crawlers that publish no list at all. There is nothing to fill in: the input schema is literally empty, `arguments: {}` and no `arguments` key at all both work, and the subject is a file this host already publishes, so the answer does not depend on you at all. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Nothing is fetched to build it — no request leaves this edge, and none is made to you. The other zero-argument call on this server is verify_my_address, same empty arguments, which answers the address you are calling from, checked against every operator-published prefix list this host mirrors with your user-agent taken as the claim — the verdict, the prefix that matched, which operator published it and when it was last mirrored. whoami and example are here too and take nothing either. Every other tool on this server wants a file pasted in; this one wants nothing. The siblings answer one question each under the tool named beside them: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/markdown (markdown_lane_self_report). Example: the complete call, exactly as written, nothing to fill in — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_

无需输入参数。

verify_my_address

Does my address verify? (no arguments)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"verify_my_address","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/netcheck and the answer is the address you are calling from, checked against every operator-published prefix list this host mirrors with your user-agent taken as the claim — the verdict, the prefix that matched, which operator published it and when it was last mirrored. THIS IS THE CALL TO MAKE FIRST on this server: it is the only one whose subject is the request you just sent, so there is nothing to look up and no argument to invent. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Every fact is computed from your own headers and from files this host already publishes; nothing is fetched to build it, and no URL of yours is touched. Only this server answers whether the address you are calling from falls inside a prefix a crawler operator publishes, and which prefix from which source — its six siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/markdown (markdown_lane_self_report) on https://www.pathwren.workers.dev; whoami is answered by all seven and is never an error, so one identical body still walks the whole family if you prefer that. Example: the complete call, exactly as written, nothing to fill in — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"verify_my_address","arguments":{}}} returns a verdict for the address you are calling from, the matching prefix with its source and mirror time, and the sentence that a miss is not evidence of a fake.

无需输入参数。

whoami

Who is calling? (no arguments)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/netcheck and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back describes YOU, the caller — the user-agent you sent, the address you came from, the class our instrument books you as, and whether this host has seen you before. Takes no arguments. Safe to call. Deterministic. Touches no third party. Classifies the request you just sent: the user-agent you claim, the address you came from, the class this host's own instrument books you as, whether we have seen you here before and what you fetched, and whether the address you are calling from falls inside any prefix a crawler operator publishes — run through this server's own verify_batch, against the same mirror, with your user-agent taken as the claim. Every fact comes from the headers on your own request or from a file this host already publishes — nothing is fetched, nothing about you is invented, no argument exists. Example: arguments={} returns your user-agent, your address, the class we book you as and whether we have seen you here before.

无需输入参数。

example

Run this server's worked example (no arguments)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/netcheck and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example — verify_batch, run for real on a canned input from this host's own data — so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — verify_batch on one address from inside a prefix Google itself publishes (mirrored here every six hours) and one RFC 5737 documentation address that is in no published range — our own mirror, no third party asked — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs verify_batch with {"addresses":[{"ip":"66.249.66.1","claim":"Googlebot"},{"ip":"203.0.113.9","claim":"GPTBot and returns its real answer.

无需输入参数。

verify_batch

Verify many addresses at once

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_report_the_crawler_ip_ranges_this_host_mirrors","arguments":{}}} — Check up to 500 addresses against every mirrored operator prefix list in one call. Returns, per address, the matching prefix, the source that published it and when it was mirrored, plus a rollup by operator. Accepts {ip, claim} pairs to test a claimed identity against the ranges, and says plainly that a miss is not evidence of a fake. Example: addresses=[{ip:'66.249.66.1', claim:'Googlebot'}, {ip:'203.0.113.9', claim:'GPTBot'}] — the first matches a published Google prefix, the second matches nothing. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/verify-crawler?ip=66.249.66.1&ua=Googlebot&s=client-dossiers

输入参数addresses必填

lookup_prefix

Prefix arithmetic against published ranges

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_report_the_crawler_ip_ranges_this_host_mirrors","arguments":{}}} — Give a CIDR and get every published operator prefix that contains it, is contained by it, or partially overlaps it — the check for 'does my network collide with a crawler range' and for auditing an allowlist somebody handed you. Give an operator or source slug instead and get everything that source publishes. Example: cidr='66.249.66.0/24'.

输入参数cidroperator

export_ip_acl

Paste-ready allowlist or denylist

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_report_the_crawler_ip_ranges_this_host_mirrors","arguments":{}}} — Turn a set of operators into a config you can paste: nginx geo, nginx allow/deny, Apache, HAProxy, a Cloudflare firewall expression, an ipset script, a Caddy matcher, or a plain CIDR list. Every export carries a provenance header naming each source URL and the mirror time, and reports the rule cost. Example: operators='all', format='cidr-list', action='allow'.

输入参数actionformatoperatorsip_versionvariable_name

verification_methods

Can this crawler be verified at all, and how?

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_report_the_crawler_ip_ranges_this_host_mirrors","arguments":{}}} — Per crawler: whether the operator publishes IP ranges, documents reverse DNS, or publishes no verification method at all — with the exact source URL or the exact dig commands, and how fresh this mirror is. Call it before reading anything into an address that matched nothing. Reverse DNS is never run here; the command is handed back instead. Example: crawler='claudebot'. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/verification-methods?crawler=claudebot&s=client-dossiers

输入参数crawler

range_stats

The state of the mirror

Takes no arguments. Safe to call. Deterministic. Touches no third party. Per source: prefix counts, total addresses and /24-equivalents, smallest and largest prefix, the publishing URL and when it was last fetched; plus prefixes published by more than one operator. This is the sizing and freshness information an allowlist decision needs. Example: {} — no arguments at all.

无需输入参数。

版本历史

1
  1. v1.4.0最新Sep 16, 2026

评分

已满足适用检查项的 83%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分

    请声明包含 README 的 GitHub 仓库,或在 OAuth 受保护资源元数据中提供 resource_documentation。

  • 许可证不适用

    托管服务按其提供方的条款使用,因此这里不要求许可证。此检查项不计入百分比。

其他 Workers 集成