AI Crawler Access Check

io.github.hamzawamiqhqv1.0.0Updated Oct 6, 2026

Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingSecurity & Monitoring

Overview

AI-generated overview

Checks whether named AI crawlers such as GPTBot, ClaudeBot and PerplexityBot can actually reach a given site.

What it does
This remote MCP server tests crawler access to a website. According to its description, it performs a real fetch per crawler rather than relying on robots.txt alone, and reports whether firewall or CDN rules block those crawlers. It is aimed at answering the question of whether GPTBot, ClaudeBot and PerplexityBot can reach a site.
When to use it
Useful when you want to confirm that AI crawlers are not being blocked before expecting a site to be indexed or cited by AI assistants. Also helpful when diagnosing why a site is not appearing in AI answers despite being publicly reachable in a normal browser.
Requirements
It runs as a remote streamable HTTP endpoint at check.citant.ai, so no local runtime or package install is needed. No authentication, environment variables or headers are declared. Network access to the endpoint is required.
Before you install
The server fetches the target site on your behalf, so the URL you submit is sent to a third-party service. No credentials, payments or write actions are declared in the manifest.

Installation

In SourceWeft

  1. Open AI Crawler Access Check in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "ai-crawler-check": {
      "type": "http",
      "url": "https://check.citant.ai/mcp?src=registry"
    }
  }
}

Tools

0
Tool metadata has not been indexed yet.

Version history

1
  1. v1.0.0LatestOct 6, 2026