Librecrawl — Technical SEO Audit MCP Server

This repository is the Sitewalk MCP server and nothing else: a Python server that drives the open-source LibreCrawl crawler, runs 30+ technical SEO checks, and hands your AI assistant a ranked report plus a zip of PDF and CSVs. It runs on your own machine, with no account, no sign-in and no page cap. Nothing leaves your machine except the requests the crawler makes to the site you audit.

Prefer not to host anything? Use the hosted ChatGPT app.

What is the ChatGPT app?

Sitewalk also runs as an app inside ChatGPT, hosted by us. There is nothing to install: open it, type Audit https://yoursite.com, and ChatGPT brings back the technical SEO problems ranked by impact, a plain fix for each, and a link to the full report. Plans and limits are on audit.adityaarsharma.com. The hosted service is separate from the code in this repository.

What is MCP?

MCP (Model Context Protocol) is an open standard that lets an AI assistant call outside tools. This repository is one such tool: your assistant asks it to crawl a site and gets the findings back. You do not need to know anything about MCP to use the ChatGPT app. It only matters if you run this repository yourself.

Set it up on your machine

You need Docker running. Then:

git clone https://github.com/adityaarsharma/sitewalk.gitcd sitewalk./setup.sh

The script checks your machine, asks one optional question (a Google PageSpeed key), builds and starts the stack, waits until the MCP server answers, and prints exactly how to connect your assistant. The first build takes a few minutes.

Without Docker, see the manual route in docs/GETTING-STARTED.md.

Connect an assistant

The server listens on http://127.0.0.1:5081/mcp (streamable HTTP).

• ChatGPT and Grok reach servers over the internet, so they need a public https address that forwards to your machine. The server has no sign-in of its own, so put it behind a tunnel that adds access control. Steps in docs/GETTING-STARTED.md.
• Clients on your computer (Codex, Windsurf, any stdio-only client) connect through mcp-remote:

{  "mcpServers": {    "sitewalk": {      "command": "npx",      "args": ["-y", "mcp-remote", "http://127.0.0.1:5081/mcp"]    }  }}

Then ask: Audit https://yoursite.com for technical SEO problems.

What you get

• 30+ checks per audit (broken links, redirects, canonicals, titles and descriptions, headings, schema, images, hreflang, sitemap gaps, and more). Full list and the nine report files: docs/CHECKS.md.
• A branded PDF, a Markdown copy and CSVs, zipped in ./reports.
• Chunked crawling that does not time out, WAF challenge detection, and a crawl delay that adapts to the site.
• 38 MCP tools: docs/TOOLS.md.

Docs

Releases

Every change to the code that lands on main is tested and published as a new GitHub release automatically. Update with ./setup.sh --update.

Licence and credits

Code is MIT: see LICENSE and NOTICE.md. Built on LibreCrawl (MIT), FastMCP and WeasyPrint. No telemetry.

Built by Aditya Sharma.