Atalaya AI crawler check

com.atalaya-aiPublished by atalaya-ai.comOfficial publisherv1.1.0Updated Nov 3, 2026

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

VerifiedStreamable HTTPWeb executableWeb Search & Scraping

Overview

Atalaya AI crawler check's hosted MCP server (atalaya-ai.com), published by atalaya-ai.com; no sign-in required; Streamable HTTP; 2 tools.

No AI overview yet: these facts come from the registry entry and the server itself.

Publisher
atalaya-ai.com
Endpoint
atalaya-ai.com
Last checked

Installation

In SourceWeft

  1. Open Atalaya AI crawler check in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/com.atalaya-ai%2Fai-crawler-check/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "atalaya-ai-crawler-check": {
      "type": "http",
      "url": "https://atalaya-ai.com/mcp"
    }
  }
}

Tools

2

check_ai_crawler_access

Check AI crawler access

Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.

Parameterslangdomainrequired

list_ai_crawlers

List known AI crawlers

Lists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that keeps AI search while opting out of training.

Takes no parameters.

Version history

1
  1. v1.1.0LatestNov 3, 2026

Score

83% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts

    Declare a GitHub repository with a README, or serve resource_documentation in the OAuth protected-resource metadata.

  • LicenseNot applicable

    A hosted service is used under its provider's terms, so no license is expected here. This check is left out of the percentage.