GEO Inspector

io.github.Bigsupe55Published by GitHub account Bigsupe55v0.1.2Updated Oct 24, 2026

Inspect a site's AI-search readiness: AI crawler access, llms.txt, schema markup, meta directives

VerifiedSTDIODesktop onlyWeb Search & ScrapingDeveloper Tools

Overview

AI-generated overview

Local MCP server that lets an assistant inspect a website's AI-search readiness: AI crawler rules, llms.txt, schema markup and meta directives.

What you can do
  • Check robots.txt rules for AI crawlers such as GPTBot, ClaudeBot and CCBot, plus sitemaps
  • Check whether a site publishes llms.txt or llms-full.txt and whether it is spec-valid
  • Detect JSON-LD blocks, schema.org types and sameAs disambiguation on a page
  • Read meta robots tags, including noai and bot-specific tags, and X-Robots-Tag headers
  • Return a readable summary plus structured JSON for each check
When to use it
Useful when you want to know whether a site allows AI crawlers to train on or cite its content, or how well it is set up for AI search and citation. It suits quick audits of a few pages rather than bulk crawling.
Requirements
Runs as a local process over stdio, started with npx from the npm package geo-inspector-mcp, so Node.js and network access are needed. No accounts, API keys or environment variables are declared.
Before you install
Each tool fetches the target site over the network, so the URLs you ask about are requested from your machine. Results reflect the live site and can change; the README notes the demo output is regenerated from real runs. No credentials or write actions are involved.

Installation

In SourceWeft

  1. Open GEO Inspector in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

geo-inspector-mcp

[npm] [MCP Registry] [license]

An MCP server that gives an AI assistant four tools for inspecting how a website presents itself to other AI systems: which AI crawlers it blocks, whether it publishes llms.txt, what schema markup it ships, and how its indexing directives are set.

Published on npm and listed in the official MCP Registry as io.github.Bigsupe55/geo-inspector-mcp. Works with Claude Code, Claude Desktop, or any MCP client.

[geo-inspector-mcp inspecting three sites]

Tool output above is verbatim from a live run against nytimes.com, docs.anthropic.com, and stripe.com, captured by driving the built server over MCP stdio. The sitemap list is collapsed to a count; nothing else is edited. Regenerate with npm run demo.

Why this exists

AI assistants are becoming a primary way people find and cite content, and sites signal their intent to AI systems through a handful of plumbing files: robots.txt rules for AI crawlers, the emerging llms.txt standard, schema.org structured data, and meta directives. Checking those by hand means juggling curl, a robots.txt parser in your head, and view-source. This server turns all of it into questions you can just ask Claude.

Quickstart

bash
npx -y geo-inspector-mcp

That is the whole install. Point your MCP client at it:

Claude Code

bash
claude mcp add geo-inspector -- npx -y geo-inspector-mcp

Claude Desktop (claude_desktop_config.json)

json
{  "mcpServers": {    "geo-inspector": {      "command": "npx",      "args": ["-y", "geo-inspector-mcp"]    }  }}

Then ask things like: "Which AI crawlers does nytimes.com block?" or "Does stripe.com publish an llms.txt?"

Tools

ToolWhat it checksExample question
check_robots_txtWhich AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and more) are allowed or blocked, per RFC 9309, plus sitemaps"Can OpenAI train on example.com?"
fetch_llms_txtPresence and spec-validity of /llms.txt and /llms-full.txt"Has example.com adopted llms.txt?"
detect_schema_markupJSON-LD blocks, schema.org type inventory, AI-relevant types, sameAs disambiguation"What structured data does this article have?"
check_meta_directivesMeta robots tags (including noai/noimageai and bot-specific tags) and X-Robots-Tag headers"Is this page indexable?"

Every tool returns a readable summary plus structured JSON (structuredContent) for programmatic use.

How it is built

The interesting part of an MCP server is not the tools, it is the contract around them.

Every tool returns two things. A readable summary for the model to reason over, and structuredContent for anything downstream that needs to compute. That split is what lets a separate scoring layer (ai-visibility-audit) derive deterministic numbers from the same call the model is reading in prose. The model never has to parse its own tool output back into data.

Parsers are pure functions. robots.txt, llms.txt, JSON-LD, and meta directives each parse in isolation, with no I/O and fixture-based tests. A tool handler is a thin shell: fetch, parse, format. This is what makes the behavior testable without a network, and it is why the test suite runs with no fixtures to record and no site to hit.

All network access goes through one helper. A single fetch path with a size cap, a redirect limit, and a timeout. An MCP server runs inside someone else's agent loop with their API budget attached, so a tool that can hang or stream an unbounded response is a tool that can ruin a session. One choke point means those limits cannot be forgotten in a new tool.

robots.txt parsing follows RFC 9309 rather than a regex, because the whole value of the tool is being right about whether a specific crawler is allowed. Longest-match wins, user-agent groups merge, and Allow can override a broader Disallow.

Development

bash
npm installnpm test        # vitest unit + integration testsnpm run build   # bundle to dist/npx @modelcontextprotocol/inspector node dist/index.js   # poke it interactively

Parsers are pure functions with fixture-based tests; all HTTP goes through one capped, redirect-limited fetch helper.

Regenerating the demo GIF

bash
npm run demo          # build, capture, render

Or the steps separately:

bash
npm run demo:capture  # drives the built server over MCP stdio, writes scripts/demo-data.jsonnpm run demo:render   # renders docs/demo.gif from that JSON (needs: pip install Pillow)

demo-data.json is committed, so rendering works offline and the GIF's claims stay auditable: diff it against what the tools return today. The capture script mocks nothing, so if a site changes its robots.txt, the demo changes with it. The only authored text in the pipeline is the human question line; the sites to inspect are configured at the top of scripts/capture-demo.mjs.

Related

ai-visibility-audit is the workflow layer on top of this one: it orchestrates these four tools into a scored, client-ready report. This repo is the raw tools.

License

MIT, see LICENSE.

Source: README.md at commit 3bdfd5d

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.1.2LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts