Web Scraper to Markdown API

io.github.Br0ski777Published by GitHub account Br0ski777v1.2.0Updated Oct 24, 2026

Extract clean markdown from any URL. Removes boilerplate. For RAG pipelines. x402.

VerifiedSSEWeb executableWeb Search & ScrapingKnowledge & Memory

Overview

AI-generated overview

Remote MCP server that scrapes web pages and returns clean markdown, with single-URL and batch tools, paid per call via x402.

What you can do
  • Scrape a single URL with JavaScript rendering and get back title, description, markdown content and word/character counts
  • Scrape up to 10 URLs in one batch call and receive per-URL results plus a summary of totals and failures
  • Strip navigation, ads, scripts and boilerplate so the output is ready as LLM input for RAG corpora and research
When to use it
Useful when an assistant needs readable page text from web sources for summarization, research or building a RAG corpus, especially across several URLs at once. It is not intended for screenshots, SEO audits, tech-stack detection or web search.
Requirements
A remote MCP endpoint added to the client configuration; no API key or signup. Payment is required per call through the x402 protocol in USDC on Base L2, so an x402-capable client and a funded wallet are needed. Network access to the endpoint.
Before you install
Every call costs money: $0.012 per single scrape and $0.05 per batch of up to 10 URLs, settled in USDC on Base L2 through a crypto wallet, so an agent can spend funds automatically. Requests send the target URLs to a third-party service, and the README points to a wider marketplace of paid APIs.

Installation

In SourceWeft

  1. Open Web Scraper to Markdown API in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via SSE. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.Br0ski777%2Fweb-scraper/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "web-scraper-to-markdown-api": {
      "type": "sse",
      "url": "https://web-scraper.api.klymax402.com/mcp"
    }
  }
}

README

Web Scraper to Markdown API

[MCP Server] [x402] [License: MIT]

Extract clean markdown from any URL. Strips nav/ads/scripts, returns structured content. Built for RAG pipelines and AI research agents. Pay-per-call via x402 (USDC on Base L2) -- no API key, no signup, no rate-limit wall.

Part of the klymax402 marketplace -- 100 x402 micropayment APIs for AI agents, one wallet, USDC on Base.

Quickstart -- MCP

Add to your MCP client config (Claude Desktop, Cursor, ElizaOS, etc.):

json
{  "mcpServers": {    "web-scraper": {      "url": "https://web-scraper.api.klymax402.com/mcp"    }  }}

Quickstart -- HTTP (x402)

bash
curl "https://web-scraper.api.klymax402.com/api/scrape?url=https://example.com"# -> 402 Payment Required, with an x402 payment challenge in the response body

Any x402-aware client (@x402/fetch, x402-agent-tools, ATXP) handles the 402 -> sign -> retry cycle automatically.

Tools

ToolMethodPathPriceDescription
web_scrape_to_markdownGET/api/scrape$0.012Scrape a URL and convert to clean markdown
web_scrape_batchPOST/api/scrape/batch$0.05Scrape up to 10 URLs in batch

web_scrape_to_markdown

Scrape and extract content from a URL with full JS rendering, returned as clean markdown. Alternative to Firecrawl scrape at 2.5x lower cost. Strips navigation, ads, scripts, and boilerplate — ideal for RAG pipelines and AI research agents.

Parameters

NameTypeRequiredDescription
urlstringyesURL to scrape (e.g. https://example.com/article)

Example response:

json
{"title":"How to Scale APIs","description":"A guide to...","content":"# How to Scale APIs\n\nScaling requires...","wordCount":1250,"charCount":7800,"url":"https://blog.example.com/scale-apis"}

When to use: summarizing articles, building RAG corpora, researching topics from web sources, or extracting data from documentation pages. Essential for any workflow that needs to scrape and extract content from web pages as LLM input. Drop-in replacement for Firecrawl scrape.

Not for: screenshots (use capture_screenshot), SEO audit (use seo_audit_page), tech stack detection (use website_detect_tech_stack), web search (use web_search_query).

web_scrape_batch

Use this when you need to extract clean content from multiple web pages at once (up to 10 URLs). Returns the same structured markdown output as web_scrape_to_markdown for each URL.

Parameters

NameTypeRequiredDescription
urlsarrayyesArray of URLs (max 10)

Example response:

json
{"results":[{"url":"https://a.com","title":"Page A","wordCount":800},{"url":"https://b.com","title":"Page B","wordCount":1200}],"summary":{"total":2,"totalWords":2000,"failed":0}}

When to use: building research corpora, comparing content across competitor pages, or bulk documentation extraction. Essential when you have 3+ URLs to process in one workflow.

Not for: single URLs (use web_scrape_to_markdown), SEO comparison (use seo_audit_batch).

Example agent prompts

  • "Scrape and extract content from a URL with full JS rendering, returned as clean markdown"
  • "Extract clean content from multiple web pages at once (up to 10 URLs)"

Payment

  • Protocol: x402 -- HTTP-native pay-per-call, no signup, no API key
  • Network: Base L2 (eip155:8453)
  • Asset: USDC
  • Facilitator: Coinbase CDP (primary), PayAI (fallback)
  • Also reachable via ATXP (OAuth-wrapped x402, RFC 9728 protected-resource metadata)

Part of klymax402

100 x402 micropayment APIs for AI agents -- one wallet, USDC on Base, zero signup.

License

MIT

Source: README.md at commit 06525b1

Tools

0
SourceWeft has not read this server's tools yet.

Version history

1
  1. v1.2.0LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declaredRequired10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts