Web Scraper & Crawler for RAG (Markdown)

io.github.ByteBliskPublished by GitHub account ByteBliskv1.0.0Updated Nov 8, 2026

Website to clean Markdown or JSON for AI and RAG: read pages, crawl sites, extract described fields.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingKnowledge & Memory

Overview

AI-generated overview

Hosted Apify MCP server that reads web pages or crawls whole sites and returns clean Markdown or JSON for AI and RAG use.

What you can do
  • Read single web pages as clean Markdown or JSON
  • Crawl whole sites and produce site maps
  • Extract fields described in plain words from pages
  • Return structured output suited to RAG pipelines
When to use it
Useful when an assistant needs web content in a clean, structured form for retrieval-augmented generation, site-wide content collection, or pulling specific fields from pages without writing a scraper.
Requirements
Remote Streamable HTTP endpoint hosted on Apify; no installation. Requires an Apify account and an Apify API token sent in the Authorization header as a Bearer token. Network access to the Apify endpoint.
Before you install
The Apify API token is a secret sent in the Authorization header. Usage is paid per use, with prices on the Store page and a monthly free-plan credit; failed calls are stated to be free. Crawling sends target site URLs to Apify's hosted service.

Installation

In SourceWeft

  1. Open Web Scraper & Crawler for RAG (Markdown) in the dashboard and add it to a workspace.
  2. Connect the required credentials. They are stored privately and never shown on public pages.
  3. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.ByteBlisk%2Fsitereader/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config. The client completes OAuth sign-in on first connect.

{
  "mcpServers": {
    "web-scraper-crawler-for-rag-markdown": {
      "type": "http",
      "url": "https://mcp.apify.com/?tools=byteblisk/sitereader"
    }
  }
}

README

byteblisk MCP servers

Three hosted MCP servers for AI agents, running on Apify. Nothing to install: connect over Streamable HTTP with your Apify API token, and pay per use. Every tool states its cost, and failed calls are free.

ServerWhat your agent getsStore page
[Image]Web Scraper & Crawler for RAGAny page or whole site as clean Markdown or JSON, site maps, fields you describe in plain wordsapify.com/byteblisk/sitereader
[Image]App Store & Google Play ReviewsReviews from both stores in one format, complaint themes with real quotes, competitor comparisonapify.com/byteblisk/app-reviews-insights
[Image]Google Play Scraper & SERP Rank CheckerGoogle Play apps, reviews and search, top-100 rank on Google, Bing and Yandexapify.com/byteblisk/google-play-search-toolkit

Connect

Endpoints (Streamable HTTP, header Authorization: Bearer <YOUR_APIFY_TOKEN>):

https://byteblisk--sitereader.apify.actor/mcphttps://byteblisk--app-reviews-insights.apify.actor/mcphttps://byteblisk--google-play-search-toolkit.apify.actor/mcp

Claude Desktop, Cursor or any MCP client (mcp.json):

json
{  "mcpServers": {    "byteblisk-sitereader": {      "url": "https://byteblisk--sitereader.apify.actor/mcp",      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }    },    "byteblisk-app-reviews": {      "url": "https://byteblisk--app-reviews-insights.apify.actor/mcp",      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }    },    "byteblisk-google-play-search": {      "url": "https://byteblisk--google-play-search-toolkit.apify.actor/mcp",      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }    }  }}

Get a token at console.apify.com/settings/integrations. Prices are on each Store page; Apify's free plan includes monthly credit to try them.

Support

Open an issue on the actor's Store page; we answer every one.

Source: README.md at commit 19fa2c2

Tools

0
This server asks you to sign in, so SourceWeft could not read its tools without an account and has not indexed them. What a connected account sees can depend on that account and its permissions.

Version history

1
  1. v1.0.0LatestNov 8, 2026

Score

75% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

    Declare a license in the repository or the registry entry.