Site Search Opener

io.github.albertalbsPublished by GitHub account albertalbsv1.1.1Updated Oct 29, 2026

SEO and competitor research from sitemaps and feeds, robots.txt respected, Google link fallback.

VerifiedSTDIODesktop onlyWeb Search & ScrapingData & Analytics

Overview

AI-generated overview

Reads public sitemaps and RSS/Atom feeds to give an assistant tables of competitor and own-site pages for SEO research, with Google search links as fallback.

What you can do
  • List and run research recipes across your own and competitor domains, returning one table of matching pages
  • Inventory a single site's pages from its sitemap and feeds, filtered by keyword, URL text or dates
  • Generate Google or Bing site: search links to open yourself when a site has no usable sitemap
  • Check domains and URLs for HTTP 200 responses and follow redirects
  • Build a project file of own and competitor domains for the companion web app
  • Save results as CSV, JSON or Markdown files on your computer (local version only)
When to use it
Useful for SEO, content, product, sales, PR, HR and link-building research where you want real page lists from many competitor sites without manual browsing. Also handy for auditing your own site's publishing activity, stale pages or URL structure.
Requirements
Runs locally over stdio via npx, needing Node.js 18.17 or newer (or a downloaded copy run with node). No account or API key. Optional SSO_OUTPUT_DIR sets where saved files go, defaulting to a Documents/Site Search Opener folder. Network access to public http and https sites is required.
Before you install
It fetches public websites on your behalf, so requests carry a SiteSearchOpenerMCP user agent and are rate-limited; robots.txt rules and opt-outs are respected. It writes files to disk when save-file is used, and project files contain your competitor lists. It does not search Google itself and sends no data to bestseo.tips.

Installation

In SourceWeft

  1. Open Site Search Opener in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

Site Search Opener MCP

Research your own site and many competitor sites from inside your AI app. Ask a question in plain words, and get a table of real pages: URL, title, description, last modified date and HTTP status.

It reads what websites publish for search engines: their sitemaps and RSS or Atom feeds. It checks robots.txt first, goes slowly, and says who it is. When a site has nothing to read, you get Google search links to open in your own browser instead.

Works with Claude (Desktop, Code and claude.ai), ChatGPT, Gemini CLI, Gemini Enterprise, OpenAI Codex, Cursor, VS Code with GitHub Copilot, Windsurf and any other app that supports MCP.

This is the AI companion to the free Site Search Opener web app by bestSEO.tips. Project files move between the two.

What you can ask

Which of my competitors have pages about maintenance backlog? Show me a table.

How is competitor.com's content organised? Which folders are biggest?

What did my competitors publish in the last 90 days?

Find case studies on rival-a.com and rival-b.com for the food industry.

Build a project file from the competitors we talked about before, and check they all work.

Which of my own pages haven't been updated in a year?

Is anyone on Reddit talking about our product category? Give me links.

How it works

  1. You ask in plain words. The AI picks a recipe, such as content-gap or case-studies, and your domains.
  2. The server reads each site's own files. robots.txt first, then the sitemaps it lists (or the usual places, like /sitemap.xml), then the site's RSS or Atom feed. Pages that robots.txt asks bots to avoid are left out.
  3. You get a table. Matching pages with URL, title, description, date and HTTP status, plus folder counts or pages per month for some recipes.
  4. No sitemap? You get links. If a site has no usable sitemap, blocks bots, or nothing matched, the server makes Google site: search links for that site. They open in your browser, the same as typing the search yourself.

What it never does:

  • It never searches Google or reads Google results. The links are for you to click.
  • It never tries to get past a CAPTCHA, a bot check or a robots.txt rule.
  • It never sends your data to bestseo.tips or anyone else. The local version runs on your computer. The hosted version runs in your own Cloudflare account.

Site owners can opt out at any time by adding this to their robots.txt:

User-agent: SiteSearchOpenerMCPDisallow: /

Install

WayForGuide
Claude Code pluginClaude Code/plugin marketplace add albertalbs/site-search-opener-mcp, then /plugin install site-search-opener@bestseo-tips
Local MCP serverClaude Desktop, Claude Code, Gemini CLI, Codex, Cursor, VS Code, WindsurfInstall guide, section B
SkillClaude, Codex, Gemini CLI, Cursor, CopilotInstall guide, section C
Hosted on Cloudflare (free)claude.ai, ChatGPT, Gemini Enterprise, teamsHosting guide

The quickest local setup: install Node.js 18.17 or newer, and point your app at:

npx -y site-search-opener-mcp

Prefer no downloads at start-up? Download this repository once and use node <folder>/plugin/server/site-search-opener.mjs instead.

Tools

ToolWhat it does
list-recipesShows a start-here list of recipes, or every recipe for a team
run-recipeRuns a recipe across your domains and returns one table, with search links for sites that have no sitemap data
site-inventoryLists one site's pages from its sitemap and feeds, with filters for keyword, URL text and dates
google-linksMakes Google (or Bing) site: search links for you to open
check-urlsChecks domains or URLs work (HTTP 200) and shows redirects
build-project-fileChecks your own and competitor domains and builds a project file for the web app
save-fileSaves a CSV, JSON or Markdown file on your computer (local version only)

Recipes

34 recipes for seven teams. Not sure where to begin? Start with one of these:

TeamStart hereWhat it finds
SEO and contentcontent-gapCompetitor pages whose URL or title matches your keyword
Productfeature-launchesLaunch and announcement pages from the last 6 months
Sales and marketingcase-studiesCase studies and customer stories, narrowed by industry if you like
PR and brandbrand-mentionsPages that mention a brand outside its own site
Customer researchforum-threadsReddit and Quora discussions on a topic
HR and talenthiring-signalsOpen roles for a company on the big job boards
Link buildingguest-post-sitesWrite-for-us pages on your topic

Everything, by team:

TeamRecipes
SEO and contentcontent gap, URL structure, publishing activity, comparison pages, gated assets, glossary pages, cannibalization, refresh candidates
Productfeature launches, integrations, pricing, release notes, docs depth, security and compliance pages
Sales and marketingcase studies, pages that name you, webinars, review sites
PR and brandnewsroom pages, brand mentions, press releases
Customer researchReddit and Quora threads, competitor help articles
HR and talentcareers pages, open roles on job boards
Link buildingguest post sites, sponsored posts, resource pages, infographics, social profiles

The full list, with what each one reads, is in docs/RECIPES.md.

Slash commands

In Claude Code with the plugin, type /site-search-opener: and pick one:

CommandUse it for
:seo :product :sales :pr :research :hr :linksResearch for that team. Add what you need, for example /site-search-opener:sales case studies food industry
:build-projectCollect your domains from this chat, memory and past chats, check them, and make a project file
:inventoryList one site's pages, for example /site-search-opener:inventory competitor.com pricing
:search-linksMake search links for a keyword
:helpA short explanation and the list of commands

Other apps show the same commands as MCP prompts, usually in a prompt or attachment menu.

Projects and the web app

A project is a named list of your own domains and competitor domains, in the same JSON format the web app uses:

json
{  "app": "sitesearch-opener",  "version": 2,  "projects": [    { "name": "HR software", "own": ["linkedin.com"], "competitors": ["indeed.com", "xing.com"] }  ]}
  • Ask the AI to build one. It can collect domains you mentioned before, check each one returns HTTP 200, and follow redirects.
  • Import it in the web app: open https://bestseo.tips/tools/site-search-opener/, go to Settings, then Export/Import Projects, then Import.
  • Export from the web app and hand the file to the AI, so it uses the same lists.

Limits

LocalHosted, Cloudflare free plan
Domains per call304
Sitemap files per domain304
Pages with title and status fetched10 per domain by default, up to 100off by default, up to 20
Domain checks per call20012

The local version waits at least one second between requests to the same site, longer if the site's robots.txt asks for a Crawl-delay.

Privacy and safety

  • The server only fetches public http and https addresses. It refuses local and private network addresses, and redirects that lead to them.
  • Requests carry the user agent SiteSearchOpenerMCP/<version> (+https://github.com/albertalbs/site-search-opener-mcp).
  • Downloads are capped in size and time.
  • Text from websites is passed to the AI as data, and the AI is told not to follow instructions inside it.

See SECURITY.md for more.

Questions

Why doesn't it just search Google for me? Google's terms do not allow automated searches, and it blocks them. Sitemaps are published so machines can read them, so that is what this tool reads. For Google results, it gives you the links.

Does it use bestseo.tips servers? No. The local version runs on your computer. The hosted version runs in your Cloudflare account. bestseo.tips only hosts the web app.

Why does a site show links instead of a table? Either the site has no sitemap or feed the tool can find, robots.txt asks bots to stay away, or nothing matched your filters. The links still let you check it yourself.

Why do I need a project file? You don't, but it saves typing the same 20 competitors every time, and the web app reads the same file.

Development

bash
npm installnpm test                 # unit tests and MCP protocol testsnpm run build            # rebuilds plugin/server/site-search-opener.mjs and copies the skillnpm run docs:recipes     # rebuilds docs/RECIPES.md from src/core/recipes.jsnpx wrangler dev         # runs the Cloudflare version locally

Layout:

src/core/      shared logic: domains, queries, recipes, robots.txt, feeds, inventory, projectssrc/server.js  the MCP server, shared by local and hosted versionssrc/stdio.js   local version (adds save-file)worker/        Cloudflare Workers versionskill/         the Agent Skillplugin/        the Claude Code plugin (commands, skill copy, bundled server)docs/          install, hosting and recipe guidestest/          tests, with a fake internet so they never touch real sites

Contributions are welcome, especially new recipes. See CONTRIBUTING.md.

License and credits

Apache License 2.0. See LICENSE and NOTICE.

If you share or build on this project, keep the NOTICE file with it, as the license asks. A link back is appreciated.

Created by Albert Adaikkalam, bestSEO.tips. LinkedIn: https://www.linkedin.com/in/albertalbs/

Third-party packages in the bundled server are listed in THIRD_PARTY_NOTICES.md.

Source: README.md at commit 9e3bf0a

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v1.1.1LatestOct 29, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts