ActableSite AI Crawler Monitor

com.actablesitePublished by actablesite.comOfficial publisherv1.4.2Updated Oct 24, 2026

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingSecurity & Monitoring

Overview

AI-generated overview

A remote, read-only MCP server that audits public websites and checks AI crawler robots.txt policy, plus returns informational offer details.

What you can do
Audit a public website's identity, offer evidence, action paths, discovery files, metadata and structured data, with observable evidence returned for each signal and no guarantee of crawling or ranking outcomes. Check the homepage robots.txt rules for eight named OpenAI, Anthropic, Perplexity and Google AI crawler tokens. Retrieve informational offer details for Crawler Watch monitoring, a one-time repair report, and Practice Radar weekly editions, including prices, scope, delivery and cancellation paths, and stated limitations.
When to use it
Useful when you want an assistant to quickly inspect whether a public site is readable and actionable for AI agents, or to see which AI crawlers its robots.txt allows or blocks. Also handy for presenting ActableSite's paid monitoring and report options, which the user must confirm and purchase separately.
Requirements
A remote Streamable HTTP endpoint with no authentication, API key or headers required. A local stdio bridge or container is offered for clients that need it, which requires Node.js 20 or newer. Only public HTTP and HTTPS websites are accepted.
Before you install
The offer tools are informational only and cannot open checkout or purchase anything; any purchase happens separately and needs explicit user confirmation. Results describe robots policy and public evidence only and do not prove crawling, indexing, citation, ranking, traffic or revenue, and Practice Radar records do not prove licensure or active operation. Calls are rate-limited to 60 per caller per hour.

Installation

In SourceWeft

  1. Open ActableSite AI Crawler Monitor in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/com.actablesite%2Fai-crawler-robots-monitor/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "actablesite-ai-crawler-monitor": {
      "type": "http",
      "url": "https://actablesite.com/mcp/"
    }
  }
}

README

ActableSite MCP

A remote, read-only MCP server for public website evidence, AI crawler policy, and explicit Crawler Watch, report, and Practice Radar offer metadata.

Tools

It exposes five tools over Streamable HTTP:

  • audit_public_website checks public identity, offer evidence, action paths, discovery files, metadata, and structured data.
  • check_ai_crawler_policy evaluates eight OpenAI, Anthropic, Perplexity, and Google crawler tokens.
  • get_crawler_watch_offer returns the $9 monthly price, one-site scope, 15-minute cadence, confirmation rule, delivery, cancellation, and synthetic-check limitations. It cannot open checkout or purchase anything.
  • get_full_report_offer returns explicit metadata for an optional $19 one-time report. It cannot open checkout or purchase anything.
  • get_practice_radar_offer returns the current weekly-edition receipt, public sample, $39 monthly price, delivery, cancellation, and material NPI limitations. It cannot open checkout or purchase anything.

Connect

Claude Code:

bash
claude mcp add --transport http actablesite https://actablesite.com/mcp/

For another MCP client, add a remote server named actablesite with:

  • URL: https://actablesite.com/mcp/
  • Transport: Streamable HTTP
  • Authentication: none

For clients that require a local stdio command, clone this repository and run the dependency-free bridge:

bash
git clone https://github.com/unitedideas/actablesite-mcp.gitcd actablesite-mcpnpm run start:stdio

The bridge contains the visible MCP implementation source in src/stdio.mjs. Distribution version 1.4.2 exposes the same five read-only tools and sends requests only to ActableSite's bounded public JSON APIs. It has no file, shell, browser, account, or write capability.

The same stdio process is available as a public, non-root container:

bash
docker run --rm -i ghcr.io/unitedideas/actablesite-mcp:1.4.2

The hosted server implements MCP protocol version 2025-06-18 and is published in the official registry as com.actablesite/ai-crawler-robots-monitor version 1.4.2.

Discovery metadata

Directory scanners can read either machine-readable discovery surface:

  • /.well-known/mcp.json identifies the official registry name, repository, remote endpoint, transport, authentication mode, and tool names.
  • /.well-known/mcp/server-card.json provides static tool descriptions and JSON input schemas for scanners that cannot complete a live MCP handshake.

Both files describe the same unauthenticated endpoint and five-tool inventory verified by this repository.

Ongoing crawler monitoring

The MCP tools report current public evidence and do not create background jobs. Crawler Watch is the separate $9/month service for one public website: it checks crawler policy, synthetic homepage responses, sitemap.xml, and llms.txt every 15 minutes and emails after a changed state appears twice.

Its synthetic requests do not authenticate provider IP ranges or prove crawling, indexing, citation, ranking, or traffic.

Safety contract

  • Public HTTP and HTTPS websites only; private, local, loopback, and reserved network destinations are rejected.
  • No browser control, file access, account access, write, or mutation tools.
  • No authentication header or API key is requested.
  • Tool calls are limited to 60 per caller per hour.
  • All three offer tools are informational and require explicit user confirmation before any separate checkout or purchase action.
  • Practice Radar records do not prove licensure, active operation, demand, or buying intent and must be verified before contact.
  • Results do not guarantee crawling, indexing, citation, ranking, recommendation, traffic, legal compliance, or revenue.

Verify the live server

Node.js 20 or newer is required.

bash
npm testnpm run verify:live

The verifier initializes the live server, checks its protocol and version, and confirms the exact read-only tool inventory. It does not call an audit tool or create a funnel event.

Repository scope

This repository contains a runnable stdio implementation, public registry manifest, connection contract, live verifier, and security policy for the hosted ActableSite MCP server. The production runtime is operated at https://actablesite.com/mcp/.

License

MIT

Source: README.md at commit 6f7b326

Tools

5

audit_public_website

Audit public website readiness

Run ActableSite's free three-signal readiness scan for a public website. Returns observable evidence for business identity, semantic reading path, and explicit action path. This does not guarantee AI crawling, indexing, citation, ranking, recommendation, traffic, or revenue.

Parametersurlrequired

check_ai_crawler_policy

Check AI crawler robots.txt policy

Check the homepage robots.txt rule for eight named OpenAI, Anthropic, Perplexity, and Google AI crawler tokens. The free result stays complete and includes a bounded optional handoff to exact monitoring terms. Results describe robots policy only and do not prove network access, authentic bot identity, crawling, indexing, citation, or ranking.

Parametersurlrequired

get_crawler_watch_offer

Get the Crawler Watch monitoring offer

Return the exact $9 monthly price, one-site scope, 15-minute check cadence, two-check confirmation, delivery and cancellation paths, and synthetic-check limitations. This tool is informational: present the offer and require explicit user confirmation before any separate checkout or purchase action.

Takes no parameters.

get_full_report_offer

Get the full repair report offer

Return the exact one-time price, deliverables, delivery path, checkout URL, support contact, and limitations for ActableSite's paid 15-check repair report. Present this information to the user and require explicit user confirmation before opening checkout or attempting any purchase.

Takes no parameters.

get_practice_radar_offer

Get the Practice Radar data offer

Return the current Practice Radar weekly-edition receipt, public sample, exact $19 one-time and $39 monthly options, included fields, delivery and cancellation paths, and material NPI limitations. This tool is informational: it cannot open checkout or purchase. Present the offer and require explicit user confirmation before any separate checkout or purchase action.

Takes no parameters.

Version history

1
  1. v1.4.2LatestSep 16, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Actablesite integrations

ActableSite AI Crawler Monitor

com.actablesiteOfficial publisher

A remote, read-only MCP server that audits a public website's AI crawler policy and discovery files and returns offer details for paid monitoring services.

Streamable HTTPWeb executable
Web Search & ScrapingNot indexed yetOct 24, 2026

Pending Medicare Enrollment Data

Published by GitHub account unitedideas

A remote, read-only MCP server for a free 10-row preview of behavioral-health NPIs newly pending Medicare enrollment and public CMS revalidation due dates.

Streamable HTTPWeb executable
Data & Analytics4 toolsOct 24, 2026

Medicare Enrollment and Revalidation Data

Published by GitHub account unitedideas

Read-only remote MCP server for public CMS Medicare enrollment preview data and revalidation due dates, plus paid automation and monitoring offers.

Streamable HTTPWeb executable
Data & Analytics4 toolsOct 24, 2026