Back to blog

MCP servers for web scraping and browser automation: 10 options compared

Ten MCP servers for scraping pages and driving a browser, compared by publisher, transport, authentication, and what each is for. Data from the SourceWeft directory, 20 November 2026.

SOSourceWeftNov 20, 20267 min read

Data as of: 20 November 2026

The short answer

  • To fetch a page as clean text or Markdown, use a hosted scraping server: Firecrawl, ScrapingBee, Apify, HasData, or Ujeebu.
  • To get past blocks and CAPTCHAs at scale, use a provider with its own proxy network: Bright Data or Apify.
  • To click, type, and log in like a person, use a browser automation server: Playwright Cloud Browsers or Browserless for a cloud browser; Playwright MCP or Chrome DevTools MCP for a browser on your own machine.

There is no single best server. The right one depends on whether you need content or interaction, and whether the browser should run remotely or locally.

How this list was chosen

This is a selection, not a ranking. Each server below met two criteria in the SourceWeft MCP directory on 20 November 2026:

  1. It is published by the company or project behind the product, under its own domain or GitHub organization, not by a third-party wrapper.
  2. Its listing is verified, meaning the registry confirmed the publisher's namespace or GitHub account.

We did not benchmark success rates, speed, or price. Where the directory could not read a server's tools, the table says so.

Scraping or browser automation?

Web scraping serverBrowser automation server
You give itA URL or a search queryA sequence of actions
You get backPage content as text, Markdown, or structured dataThe state of a live page after each action
Good forResearch, monitoring, feeding documents to a modelLogged-in sites, forms, multi-step flows, testing
Cost driverRequests and proxy useBrowser session time

If a plain fetch gets you the content, use a scraping server. It is cheaper and less fragile. Reach for a browser only when the page needs interaction.

Web scraping servers

ServerPublisherRunsAuthenticationTools listedUpdated
FirecrawlGitHub firecrawlRemoteNone for scrape, search and parse; API key or OAuth for the rest320 Nov 2026
Apify MCP Serverapify.comRemoteOAuth or API keySign-in required to read20 Nov 2026
ScrapingBeescrapingbee.comRemoteAPI key1826 Oct 2026
HasData Web Scrapinghasdata.comRemoteAPI keyNot indexed yet24 Oct 2026
Ujeebuujeebu.comRemoteNone required; optional API key1710 Nov 2026
Bright DataGitHub brightdataLocalSet locallyNot validated yet11 Nov 2026

"Remote" means Streamable HTTP. "Local" means stdio: the server runs on your machine and needs a desktop host. "Set locally" means the MCP connection needs no sign-in, and you give the server its API token in local configuration.

Firecrawl. Searches, scrapes, and interacts with web pages, and returns Markdown or structured data. The hosted endpoint works without a credential for scraping, search, and document parsing, with rate limits. Crawling, site mapping, and the other tools need an API key or an OAuth sign-in.

Apify MCP Server. Lets an assistant find, run, and read results from the scraping and automation Actors in the Apify Store. Choose it when you need a ready-made scraper for a specific site rather than a general fetcher.

ScrapingBee. Scrapes pages, runs web and marketplace searches, and returns structured data through the ScrapingBee API. A straightforward choice for "give me this page".

HasData Web Scraping. Fetches public pages through managed proxies, with optional JavaScript rendering and extraction rules. The same publisher lists separate servers for specific sources such as Airbnb, Google Hotels, and Zillow.

Ujeebu. Combines page scraping, Google results, and Markdown conversion in one remote server.

Bright Data. Searches the web, scrapes pages as Markdown, runs structured scrapers, and can drive a remote browser. The directory lists the local package, which needs a desktop host. Bright Data also hosts a remote endpoint. Both need a Bright Data API token.

Browser automation servers

ServerPublisherRunsAuthenticationTools listedUpdated
Playwright Cloud Browsersmicrosoft.comRemoteMicrosoft Entra IDNot indexed yet29 Oct 2026
Browserlessbrowserless.ioRemoteOAuth or API key2119 Nov 2026
Playwright MCPGitHub microsoftLocalNoneNot validated yet19 Nov 2026
Chrome DevTools MCPGitHub ChromeDevToolsLocalNoneNot validated yet11 Nov 2026

Playwright Cloud Browsers. Microsoft's remote server for Playwright Workspaces, in preview. It gives an agent managed cloud browsers: create a session, navigate, inspect the page, and interact with elements. It needs an Azure subscription and a Playwright workspace, and signs in with Microsoft Entra ID.

Browserless. The official Browserless server for browser automation, scraping, screenshots, and PDFs.

Playwright MCP. Microsoft's Playwright tools as a local MCP server. It drives a browser on your own machine, so logged-in sessions stay there.

Chrome DevTools MCP. Lets a coding agent control and inspect a live Chrome through DevTools, including network and console debugging and performance traces. It drives the browser on your own machine, which suits debugging your own app more than scraping other sites. It sends usage statistics to Google by default, which you can turn off with --no-usage-statistics.

Browserbase also publishes an MCP server. Its open-source repository is archived, and Browserbase recommends its hosted server at mcp.browserbase.com instead. The SourceWeft directory listed only the archived repository on 20 November 2026, so it is not in the table.

How to choose

  1. Content or interaction? Content → scraping server. Interaction → browser server.
  2. Remote or local? Web-based AI apps can only use remote (Streamable HTTP) servers. Local (stdio) servers need a desktop host.
  3. Is the target site hard to reach? Sites with strong bot protection need a provider with a proxy network.
  4. Do you need a logged-in session? Then you need a browser server, and you should think about where those credentials live. A local browser keeps them on your machine.
  5. What will it cost? The hosted options are commercial services with their own plans. Each listed a free plan or a free trial on 20 November 2026. Check the provider's pricing before building on one.

Use them responsibly

A scraping tool does what the model asks. Before pointing one at a site, check the site's terms of service and robots.txt, keep request rates low, and do not collect personal data you have no basis to process. Pages a tool returns can also contain text aimed at the model, so treat scraped content as data, not instructions.

Browse more

The SourceWeft directory groups these under two categories: Web Search & Scraping and Browser Automation. Each server page shows its publisher, transport, authentication, and tool list where one could be read.

FAQ

What is the best MCP server for web scraping? It depends on the job. For fetching single pages as Markdown, a hosted scraping server such as Firecrawl, ScrapingBee, or HasData is enough. For site-specific scrapers, Apify has a large catalogue. For heavily protected sites, use a provider with its own proxy network, such as Bright Data.

What is the best MCP server for browser automation? For a cloud browser, look at Playwright Cloud Browsers and Browserless. For a browser on your own machine, use Playwright MCP or Chrome DevTools MCP.

Is there a free MCP server for web scraping? Local open-source servers cost nothing to run, but you supply the machine and the network, and they are blocked more easily. The six hosted services listed here are commercial products. On 20 November 2026 each one's pricing page offered a free plan or a free trial.

Can an MCP server log in to websites for me? A browser automation server can, because it drives a real browser. Decide carefully where the session runs: a cloud browser holds your logged-in session on someone else's infrastructure.

Do I need a browser server to scrape JavaScript-heavy pages? Not always. Several scraping servers render JavaScript on their side and return the final content.

Is web scraping with an AI agent legal? It depends on the site, the data, and your jurisdiction. Public pages are not automatically free to collect. Check the site's terms and any personal-data rules that apply to you. This is not legal advice.

Shareable URL

Keep reading

Related essays

All posts

How to install skills and MCP servers in Claude Code, from an empty folder

Start from an empty folder and install a skill and an MCP server for Claude Code: search, inspect before installing, where the config files go, and how to check and remove them. Every command was actually run.

Read next

MCP servers by category: the 17 categories in our directory and how to browse them

A list of MCP servers sorted into 17 categories, from databases to browser automation to communication, with the number of servers in each as of 21 November 2026, and how to filter, search and install from a category page.

Read next

What is the Model Context Protocol (MCP)?

MCP is an open protocol that lets AI applications connect to tools and data through one standard interface. How it works, what a server exposes, and how to start.

Read next