Mcp Site Crawler Links Tables

com.mambabuilt由 mambabuilt.com 发布v1.0.0更新于 Nov 7, 2026

Crawl a website for links, file download links, HTML tables as rows, and lists, with no AI

已验证STDIO仅桌面网页搜索与抓取数据与分析

概览

AI 生成的概览

通过 Apify actor 爬取网站，按页面返回链接、下载链接、HTML 表格和列表的行数据。

可以做什么
  • 从一个或多个起始地址爬取站点，跟随链接并可选用 sitemap
  • 按页面收集内部链接和外部链接，包括指向其他域名的链接
  • 按扩展名列出文件下载链接，但不下载文件
  • 把每个 HTML 表格和每个列表提取为行
  • 通过页数上限、深度、仅同域名和 robots.txt 设置控制爬取范围
适用场景
适合需要结构化盘点站点链接、可下载文件、表格或列表的场景，例如审查外链或从多个页面提取表格数据。由于不运行浏览器，不适合 JavaScript 渲染的页面。
运行要求
以 npx 运行的本地进程，需要 Node.js 18 或更高版本。需要 Apify 账号，并通过环境变量 APIFY_TOKEN 提供 Apify API 令牌；无令牌时仍会列出工具，但调用需要令牌。需要访问 Apify 的网络连接。
安装前请注意
运行按事件计费并计入你的 Apify 账号（页面行、住宅代理页面行、下载链接行、actor 启动），爬取大站点或使用住宅代理会产生费用，可用 max_charge_usd 限制。APIFY_TOKEN 是机密凭据。爬取会把目标网址发送给 Apify；obey_robots 可被关闭，仅应在你有权这样做时关闭。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Mcp Site Crawler Links Tables，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Desktop only，通过 STDIO。 STDIO 服务会启动本地进程，因此需要 SourceWeft 桌面宿主。

其他 MCP 客户端

参照 仓库 中的启动说明。

README

Site Crawler for Links and Tables MCP Server

[npm version] [license]

An MCP server for the Mamba Labs Site Crawler for Links and Tables actor on Apify. Crawl a website and get its links, download links, tables, and lists as rows.

What's Inside

What it does

Give a start address. The actor crawls the site over plain HTTP, following links and the sitemap, and returns one row per page with internal links, external links, file download links, every HTML table as rows, and every list. The output is deterministic, with no AI. robots.txt is obeyed by default. Pages built by JavaScript return little, because no browser runs.

All of the work runs on Apify. This package is a thin client that starts the actor run, waits for it, and hands back the dataset rows.

Quick start

You need Node.js 18 or newer and an Apify account with an API token.

Add this to your Claude Desktop config (claude_desktop_config.json), Cursor, or any MCP client:

json
{  "mcpServers": {    "mamba-site-crawler-links-tables": {      "command": "npx",      "args": [        "-y",        "@mambalabsdev/mcp-site-crawler-links-tables"      ],      "env": {        "APIFY_TOKEN": "your-apify-token"      }    }  }}

Restart the client. The crawl_site_links_tables tool appears.

Prerequisites

Example prompts

  • "List every external download link on this website."
  • "Pull all the tables from these pages into rows."
  • "Crawl this site and show me which pages link out to other domains."

Tool

crawl_site_links_tables. One tool. Example arguments:

json
{  "start_urls": [    "https://webscraper.io/test-sites/tables"  ],  "extract": [    "tables",    "links"  ],  "max_pages": 10,  "max_depth": 1}

Inputs

Every buyer facing input of the actor. Omitted inputs take the actor's own defaults.

InputTypeRequiredDescription
start_urlsarraynoWeb addresses to start from, up to 100. A bare domain such as example.com is read as https.
max_pagesnumbernoPages to read across the whole run, up to 10,000. Default 100.
max_depthnumbernoHow many links to follow from a start address. 0 reads only the start address. Default 10.
same_domain_onlystringnoyes (default): follow links only on the start address's host and its subdomains. no: also follow links to other sites, up to Maximum pages. Links to other sites are listed either way.
extractarraynoAny of links, tables, lists, download_links. Default links, tables, lists. download_links adds one separate row per file link, charged as download-link-row; the download_links field on page rows is always filled at no extra charge.
download_extensionsarraynoFile extensions that count as download links. Default pdf, zip, xlsx, xls, docx, doc, pptx, csv, mp3, mp4, exe, dmg, msi, pkg, gz, tar, tgz, 7z, rar, iso, apk. Files are listed, never downloaded.
use_sitemapstringnoyes (default): also queue the addresses in /sitemap.xml, which finds pages no link reaches. no: follow links only.
obey_robotsstringnoyes (default): skip pages robots.txt disallows and honor its crawl delay up to 5 seconds. no: read them, only if you have the right to.
max_charge_usdstringnoOptional. The run stops before a charge that would pass this amount. Summary rows are always returned.
proxy_modestringnopool (default): the Apify proxy pool only, charged as page-row; a page that refuses it gets a free blocked row. auto: a refused page is retried on residential addresses, and a page read that way is charged as residential-page-row. residential: every page uses residential addresses and is charged as residential-page-row.

Pricing

Pay per event on Apify, billed to your Apify account. Prices in USD.

EventFree planBronzeSilverGold
page-row, one page read on the proxy pool$0.006$0.0057$0.0054$0.0051
residential-page-row, one page read on residential addresses, only with proxy_mode auto or residential$0.02$0.019$0.018$0.017
download-link-row, one separate file link row, only when extract includes download_links$0.0001$0.000095$0.00009$0.000085
apify-actor-start, per run start$0.0055$0.0055$0.0055$0.0055

Failed or refused pages, summary rows, pages skipped by robots.txt, and refused input are free. Mamba Labs pays the platform costs of the run, Apify proxy traffic included. Mamba Labs pays the platform costs of the run, Apify proxy traffic included.

Full actor documentation

apify.com/mambalabs/site-crawler-links-tables

License

MIT. See LICENSE.

来源：README.md，提交 1504c1a

工具

0
SourceWeft 尚未在沙箱中启动过该软件包，因此其工具未知。

版本历史

1
  1. v1.0.0最新Nov 7, 2026

评分

已满足适用检查项的 50%

该评分衡量此收录条目的完整程度，依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价，也不表示服务器安全或当前可用：可连通性是带日期的检查结果，评分也不包含安装量或用户评价。

  • 近 30 天内可连通25 分

    请保持端点能够响应 MCP 握手；SourceWeft 每 30 天探测一次。

  • 已声明认证方式10 分

    请在注册中心条目中声明凭据请求头，或保持端点能够响应探测，以便判定其认证方式。

  • 已列出工具15 分

    请让服务器在无凭据时响应 tools/list，或发布可在沙箱中无凭据启动的软件包。

  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分