Scrapiq

io.scrapiqPublished by scrapiq.ioOfficial publisherv0.2.0Updated Oct 24, 2026

Any URL to clean text, markdown, or JSON for LLM/RAG pipelines.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingKnowledge & Memory

Overview

AI-generated overview

Scrapiq fetches a web page and returns clean text, markdown or JSON, so an assistant can turn any URL into content for LLM or RAG pipelines.

What you can do
  • Extract a web page into clean structured content, choosing markdown, text or JSON output
  • Strip boilerplate, navigation, ads and scripts from the fetched page
  • Return the page title, content, links and metadata
  • Truncate the extracted content to a chosen character limit
When to use it
Useful when an assistant needs readable page content rather than raw HTML, for example to feed documents into a RAG pipeline, summarize an article, or collect links and metadata from a URL.
Requirements
A remote streamable HTTP endpoint at scrapiq.io, or a local install run from the project repository with a running Scrapiq instance. Local use needs the SCRAPIQ_ENDPOINT environment variable pointing at that instance. The hosted server needs no key.
Before you install
The hosted server sends every requested URL to a third party for fetching. Locally, the server is stateless and CORS is open, so anyone who can reach the HTTP endpoint can call it. Extraction results come from arbitrary web pages and should be treated as untrusted content.

Installation

In SourceWeft

  1. Open Scrapiq in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.scrapiq%2Fscrapiq/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "scrapiq": {
      "type": "http",
      "url": "https://scrapiq.io/mcp"
    }
  }
}

README

Scrapiq MCP Server

MCP server for Scrapiq — turn any URL into clean text, markdown, or JSON for LLM/RAG pipelines, directly from your MCP client.

Scrapiq is a lightweight open-source HTTP API that fetches a web page and returns clean content — boilerplate stripped. This server exposes it as a Model Context Protocol (MCP) tool so Claude Desktop, Cursor, and any MCP client can extract clean web content with one call.

Dependency-free: pure Python stdlib, no pip packages, no node_modules. Two transports: stdio for local clients, streamable HTTP for remote clients.

Use the hosted server (no install)

The server is listed in the official MCP Registry as io.scrapiq/scrapiq and runs at:

https://scrapiq.io/mcp

No key, no install — add that URL as a remote MCP server in any client that supports streamable HTTP:

json
{  "mcpServers": {    "scrapiq": { "url": "https://scrapiq.io/mcp" }  }}

Install locally

Not on PyPI yet, so install straight from this repo:

bash
# run it without installing anything (run against `main`, 2026-09-17)uvx --from git+https://github.com/NG-PR0JECT/scrapiq-mcp-server scrapiq-mcp
# or install it into an isolated environmentpipx install git+https://github.com/NG-PR0JECT/scrapiq-mcp-server

Requires a running Scrapiq instance (see Scrapiq README — git clone, pip install -e ".[dev]", then scrapiq). Point the server at it:

bash
SCRAPIQ_ENDPOINT=http://localhost:8001/v1/extract scrapiq-mcp

Run your own HTTP endpoint

bash
scrapiq-mcp --http --host 127.0.0.1 --port 8002   # serves POST /mcp

Stateless: one POST /mcp per JSON-RPC message (or batch), replies with application/json. It issues no Mcp-Session-Id and offers no server→client SSE stream, so GET /mcp answers 405 by design. CORS is open, so browser-based clients (e.g. MCP Inspector) can call it directly.

Usage with Claude Desktop

Add to claude_desktop_config.json:

json
{  "mcpServers": {    "scrapiq": {      "command": "scrapiq-mcp",      "env": { "SCRAPIQ_ENDPOINT": "http://localhost:8001/v1/extract" }    }  }}

Tool

scrapiq_extract

Extract a web page into clean structured content.

Arguments:

  • url (string, required) — the URL to extract
  • format (string, optional) — "markdown" (default) | "text" | "json"
  • max_chars (integer, optional) — truncate content to N chars

Example:

scrapiq_extract(url="https://en.wikipedia.org/wiki/Retrieval-augmented_generation", format="markdown")

Returns title, content, links, and metadata — no ads, no nav, no scripts.

Test the server

bash
printf '%s\n' \  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05"}}' \  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"scrapiq_extract","arguments":{"url":"https://example.com","format":"text"}}}' \  | scrapiq-mcp

License

MIT

Source: README.md at commit d9dcde1

Tools

1

scrapiq_extract

Extract a URL to clean content

Extract a web page into clean, structured content for LLM/RAG pipelines via the Scrapiq API. Strips boilerplate, navigation, ads, and scripts. Returns title, content, links, and metadata.

Parametersurlrequiredformatmax_chars

Version history

1
  1. v0.2.0LatestSep 19, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts