Webscrape Mcp

io.github.carrasquelalex1Published by GitHub account carrasquelalex1v1.0.0Updated Oct 24, 2026

Fetch any URL and get clean Markdown. Web scraping for AI agents.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingBrowser Automation

Overview

AI-generated overview

A remote web-scraping MCP server that lets an assistant fetch URLs as clean Markdown, search the web, batch-fetch pages and capture screenshots.

What you can do
Fetch a single URL and get clean Markdown, with optional readability mode, JavaScript rendering, structured-data extraction and automatic PDF text extraction. Batch-fetch up to five URLs in parallel with the same options. Search the web across DuckDuckGo, Google and Bing and scrape the results into Markdown. Capture PNG or JPEG screenshots of pages with configurable viewport and full-page mode.
When to use it
Useful when an assistant needs readable page content, search results or screenshots from the public web, including JavaScript-heavy sites and PDFs, without building its own scraping pipeline.
Requirements
A remote endpoint; the manifest lists no authentication, while the README's hosted option uses an API key sent as an Authorization Bearer header. A local install instead needs Python, the listed dependencies and a Playwright Chromium download. Network access is required.
Before you install
The server fetches arbitrary URLs on the assistant's behalf, so pages it retrieves reach a third-party service. The README's hosted option requires an API key in the Authorization header; keep it private. Screenshots and fetched content may include private or paywalled material if pointed at such URLs.

Installation

In SourceWeft

  1. Open Webscrape Mcp in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.carrasquelalex1%2Fwebscrape-mcp/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "webscrape": {
      "type": "http",
      "url": "https://webscrape-mcp.onrender.com/"
    }
  }
}

README

WebScrape MCP Server

English · Español

English

MCP server that lets AI agents search the web and extract clean Markdown content — no ads, no clutter, just the text your LLM needs.

What's New in v2.0.0

New features:

  • JS rendering — Render JavaScript-heavy sites with Playwright (Chromium headless). Auto-fallback when httpx gets 403 or empty content
  • Structured data extraction — Extract JSON-LD, Open Graph, Twitter Cards, meta tags, canonical URLs, and hreflang links with extract_schema=True
  • Screenshots — New webscrape_screenshot tool captures page screenshots with configurable viewport, full-page mode, and PNG/JPEG format
  • Multi-engine search — DuckDuckGo primary, automatic fallback to Google and Bing if DDGS is unavailable
  • Smart truncation — Content is truncated at paragraph/sentence boundaries instead of mid-word

Improvements:

  • Enhanced cache: 500 entries with 15-minute TTL (was 200, no expiry)
  • Better error handling with specific messages for 403, 404, 429, timeouts
  • Updated Dockerfile with Chromium dependencies for Playwright

Tools

ToolDescription
webscrape_fetch_urlFetch a single URL and return clean Markdown. Supports use_readability, js_render, extract_schema, and auto-detects PDFs
webscrape_batch_fetchFetch up to 5 URLs in parallel. Supports PDF auto-detection, JS rendering, and structured data
webscrape_searchSearch the web (DuckDuckGo → Google → Bing fallback) and scrape results into Markdown
webscrape_screenshotCapture a screenshot of any web page with headless Chromium. Supports PNG/JPEG, viewport sizing, and full-page capture

Features

  • PDF support: URLs ending in .pdf or with application/pdf content-type are auto-detected and text is extracted page by page
  • Readability mode: Pass use_readability=True to webscrape_fetch_url for cleaner article extraction using Mozilla Readability (removes nav, sidebars, ads, comments)
  • JS rendering: Pass js_render=True to render JavaScript-heavy sites with Playwright (headless Chromium). Auto-fallback when httpx gets 403 or empty content
  • Structured data extraction: Pass extract_schema=True to extract JSON-LD, Open Graph, Twitter Cards, meta tags, canonical URLs, and hreflang links
  • Multi-engine search: DuckDuckGo primary, automatic fallback to Google and Bing if DDGS is unavailable
  • Screenshots: Capture page screenshots with configurable viewport, full-page mode, and PNG/JPEG format
  • Built-in cache: 500-entry cache with TTL-based eviction (15 min) for repeated URLs
  • Batch fetching: Up to 5 URLs in parallel
  • Smart truncation: Content is truncated at paragraph/sentence boundaries instead of mid-word

How to use

Option 1 — MCPize (recommended)
  1. Go to https://mcpize.com/marketplace
  2. Search Web Scrape and click Start Free
  3. You'll get an API key
  4. Configure in your AI client:
json
{  "mcpServers": {    "webscrape": {      "url": "https://webscrape.mcpize.run",      "headers": {        "Authorization": "Bearer your-api-key"      }    }  }}
Option 2 — Render (dev)
json
{  "mcpServers": {    "webscrape": {      "url": "https://webscrape-mcp.onrender.com"    }  }}
Option 3 — Local
bash
git clone https://github.com/carrasquelalex1/webscrape-mcp.gitcd webscrape-mcppip install -r requirements.txtplaywright install chromiumpython webscrape_mcp.py

Official Registry

io.github.carrasquelalex1/webscrape-mcp

Dependencies

mcp, httpx, beautifulsoup4, markdownify, pydantic, ddgs, readability-lxml, PyMuPDF, playwright

License

MIT

Español

Servidor MCP que permite a agentes de IA buscar en la web y extraer contenido limpio en Markdown — sin anuncios, sin navegación, solo el texto que tu LLM necesita.

Novedades en v2.0.0

Nuevas características:

  • Rendering JS — Renderiza sitios con JavaScript usando Playwright (Chromium headless). Fallback automático cuando httpx recibe 403 o contenido vacío
  • Extracción de datos estructurados — Extrae JSON-LD, Open Graph, Twitter Cards, meta tags, URLs canónicas y links hreflang con extract_schema=True
  • Capturas de pantalla — Nueva herramienta webscrape_screenshot que captura screenshots con viewport configurable, modo full-page y formato PNG/JPEG
  • Búsqueda multi-motor — DuckDuckGo primario, fallback automático a Google y Bing si DDGS no está disponible
  • Truncado inteligente — El contenido se trunca en límites de párrafo/oración en vez de cortar palabras a la mitad

Mejoras:

  • Caché mejorada: 500 entradas con TTL de 15 minutos (antes 200, sin expiración)
  • Mejor manejo de errores con mensajes específicos para 403, 404, 429, timeouts
  • Dockerfile actualizado con dependencias de Chromium para Playwright

Tools

ToolDescripción
webscrape_fetch_urlObtiene una URL y la convierte a Markdown limpio. Soporta use_readability, js_render, extract_schema, y detecta PDFs automáticamente
webscrape_batch_fetchObtiene hasta 5 URLs en paralelo. Soporta detección de PDFs, rendering JS, y datos estructurados
webscrape_searchBusca en la web (DuckDuckGo → Google → Bing como fallback) y extrae los resultados a Markdown
webscrape_screenshotCaptura una captura de pantalla de cualquier página web con Chromium headless. Soporta PNG/JPEG, tamaño de viewport, y captura completa

Características

  • Soporte PDF: URLs que terminan en .pdf o con content-type application/pdf se detectan automáticamente y se extrae el texto página por página
  • Modo Readability: Usá use_readability=True en webscrape_fetch_url para extraer artículos de forma más limpia (elimina navegación, barras laterales, anuncios, comentarios)
  • Rendering JS: Usá js_render=True para renderizar sitios con JavaScript usando Playwright (Chromium headless). Fallback automático cuando httpx recibe 403 o contenido vacío
  • Extracción de datos estructurados: Usá extract_schema=True para extraer JSON-LD, Open Graph, Twitter Cards, meta tags, URLs canónicas, y links hreflang
  • Búsqueda multi-motor: DuckDuckGo primario, fallback automático a Google y Bing si DDGS no está disponible
  • Capturas de pantalla: Captura de páginas con viewport configurable, modo full-page, y formato PNG/JPEG
  • Caché integrada: 500 entradas con evicción por TTL (15 min) para URLs repetidas
  • Batch fetching: Hasta 5 URLs en paralelo
  • Truncado inteligente: El contenido se trunca en límites de párrafo/oración en vez de cortar palabras a la mitad

Cómo usarlo

Opción 1 — MCPize (recomendada)
  1. Ve a https://mcpize.com/marketplace
  2. Busca Web Scrape y haz clic en Start Free
  3. Obtendrás una API key
  4. Configura en tu cliente de IA:
json
{  "mcpServers": {    "webscrape": {      "url": "https://webscrape.mcpize.run",      "headers": {        "Authorization": "Bearer tu-api-key"      }    }  }}
Opción 2 — Render (desarrollo)
json
{  "mcpServers": {    "webscrape": {      "url": "https://webscrape-mcp.onrender.com"    }  }}
Opción 3 — Local
bash
git clone https://github.com/carrasquelalex1/webscrape-mcp.gitcd webscrape-mcppip install -r requirements.txtplaywright install chromiumpython webscrape_mcp.py

Registro Oficial

io.github.carrasquelalex1/webscrape-mcp

Dependencias

mcp, httpx, beautifulsoup4, markdownify, pydantic, ddgs, readability-lxml, PyMuPDF, playwright

Licencia

MIT

Source: README.md at commit 3895cf2

Tools

0
SourceWeft has not read this server's tools yet.

Version history

1
  1. v1.0.0LatestSep 16, 2026

Score

75% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

    Declare a license in the repository or the registry entry.