Sitemap URL Extractor for AI Agents

io.github.industrial-platform-aiv1.0.2更新于 Sep 30, 2026

x402 sitemap discovery and URL extraction for agents. $0.0001 per emitted URL.

已验证Streamable HTTP可网页运行Web Search & ScrapingData & Analytics

概览

AI 生成的概览

让助手发现并展开公开站点地图，输出干净的 URL 行，按输出的 URL 计费。

功能
这个托管 MCP 端点提供 Sitemap Intelligence 服务，可发现站点地图并将其展开为干净的 URL 行。它是通过同一 Apify 托管端点访问的多个付费 Industrial Platform 服务之一，工具发现免费，仅在调用付费 Actor 时收费。结果以结构化的机器可读格式返回。
适用场景
当助手需要从网站站点地图获取 URL 列表，用于抓取、索引或研究流程，且委托该提取比自己构建更简单时使用。它适合已经在调用其他 Industrial Platform 服务的代理工作流。
运行要求
需要一个指向托管 Apify 端点的远程 MCP 客户端，无需本地软件包。付费执行需要受支持的身份验证或代理支付方式，例如 Apify OAuth、Apify API 令牌，或使用已充值钱包的 x402 支付。需要能访问该端点的网络。
安装前请注意
调用按输出的 URL 计费，成本随返回的 URL 数量增加；执行前请核实当前价格与资格。付费执行需要凭据或钱包资金，README 提醒不要提交 API 令牌、钱包凭据、机密或私钥。该服务会抓取任意公开 URL，请了解其对私有或本地网络目标的处理方式。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Sitemap URL Extractor for AI Agents，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable，通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "sitemap": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=industrial_platform/sitemap-intelligence&payment=x402"
    }
  }
}

README

Industrial Platform Agent Tools

Open-source integration toolkit for connecting AI agents, MCP clients, autonomous workflows, agent orchestrators, and developer applications to Industrial Platform machine-callable services.

Industrial Platform provides specialized paid utilities that other agents can discover and invoke instead of rebuilding research, monitoring, extraction, and document-processing capabilities inside every workflow.

One MCP endpoint for the full portfolio

text
https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402

This hosted MCP endpoint exposes the six currently runnable Industrial Platform services through one direct x402 integration. Tool discovery is free; autonomous callers pay only when invoking a paid Actor.

Current services

ServiceActorPrimary purposeBilling eventCurrent price
AI Research Briefindustrial_platform/research-brief-agentCurrent, cited multi-source researchresearch-brief-completed$1.99
Web Change Intelligenceindustrial_platform/web-change-intelligenceDetect deterministic changes in public web contentpage-comparison$0.01
Web Metadata Intelligenceindustrial_platform/web-metadata-intelligenceExtract page metadata, OG/Twitter, JSON-LD, headingsmetadata-extraction$0.01
Article Content Intelligenceindustrial_platform/article-content-intelligenceExtract clean article text for RAG/research pipelinesarticle-extraction$0.002
PDF Text Intelligenceindustrial_platform/pdf-text-intelligenceExtract PDF text, per-page text, and metadatapdf-extraction$0.0015
Sitemap Intelligenceindustrial_platform/sitemap-intelligenceDiscover and expand sitemaps into clean URL rowssitemap-url$0.0001
Link Intelligenceindustrial_platform/link-intelligenceExtract structured page links, anchor text, rel flags, and internal/external classificationlink-extraction$0.0005

Live pricing and execution eligibility should be verified against Apify Store before paid execution.

Official MCP Registry

Industrial Platform is published in the official MCP Registry as:

text
io.github.industrial-platform-ai/industrial-platform

Registry version: 1.0.1 — publication automated on merge.

The Registry listing points to the same hosted seven-service Apify MCP bundle used by this repository.

Individual MCP Registry entries

For search-oriented discovery, each paid service also has its own active MCP Registry entry:

ServiceMCP Registry name
Research Briefio.github.industrial-platform-ai/research-brief
Web Changeio.github.industrial-platform-ai/web-change
Web Metadataio.github.industrial-platform-ai/web-metadata
Article Extractorio.github.industrial-platform-ai/article-extractor
PDF Textio.github.industrial-platform-ai/pdf-text
Sitemapio.github.industrial-platform-ai/sitemap
Link Extractorio.github.industrial-platform-ai/link-extractor

These individual entries point to the same Apify-hosted Actor tools as the portfolio bundle and improve discovery for capability-specific MCP searches.

Machine-readable discovery

The repository publishes:

text
catalog.jsonmanifests/mcp-bundle.jsonexamples/mcp/remote-server.json

Service catalog

catalog.json is the portfolio-level machine-readable catalog. It contains Actor IDs, Store URLs, billing events, current prices, and common delegation targets.

MCP bundle manifest

manifests/mcp-bundle.json defines the hosted MCP endpoint and the Actors included in the bundle.

MCP client configuration

examples/mcp/remote-server.json can be used as a starting point for MCP clients that accept remote server configuration.

Agent-to-agent paid quick start

bash
npm install -g @apify/mcpcmcpc x402 initmcpc connect "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402" @industrial-platform --x402mcpc @industrial-platform tools-list

Fund the generated wallet with USDC on Base before paid calls. Product #7 (Link Intelligence) is not included here until its public/agentic publication is confirmed.

Quick start

MCP

Configure one remote MCP server:

json
{  "mcpServers": {    "industrial-platform": {      "url": "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence"    }  }}

Compatible clients can then discover the exposed Actor tools from that server.

REST / Apify API

Every service can also be invoked directly through Apify's Actor API.

General Actor identifier form:

text
industrial_platform~ACTOR-NAME

Example:

text
industrial_platform~pdf-text-intelligence

Do not commit API tokens, wallet credentials, secrets, or private keys to this repository.

When should an agent delegate?

A calling agent should prefer a specialized Industrial Platform service when delegation is cheaper, simpler, or operationally safer than rebuilding the capability internally.

Examples:

text
Need current multi-source research with citations→ research-brief-agent
text
Need to know whether a monitored page changed→ web-change-intelligence
text
Need Open Graph, JSON-LD, canonical, robots, and headings→ web-metadata-intelligence
text
Need clean article text for a RAG or research pipeline→ article-content-intelligence
text
Need text and metadata from a public PDF→ pdf-text-intelligence
text
Need structured links, anchor text, and internal/external classification→ link-intelligence

Portfolio architecture

text
developer / agent / orchestrator             ↓   Industrial Platform MCP             ↓  specialized machine service             ↓   structured result / event             ↓calling system continues workflow

Execution and monetization currently run through Apify.

Authentication and machine payments

Paid execution requires a supported authenticated or agentic-payment path.

Depending on the integration, this may use:

  • Apify OAuth
  • Apify API tokens
  • Apify-supported agentic payment infrastructure

Store eligibility and pricing can change. Calling systems should inspect current Actor metadata before executing paid services.

Repository structure

text
industrial-platform-agent-tools/│├── README.md├── catalog.json├── AUTONOMOUS_REVENUE_INITIATIVE.md│├── actors/│   ├── web-change-intelligence/│   ├── web-metadata-intelligence/│   ├── article-content-intelligence/│   ├── pdf-text-intelligence/│   ├── sitemap-intelligence/│   └── link-intelligence/│├── packages/│   ├── research-mcp/│   └── change-mcp/│├── manifests/│   └── mcp-bundle.json│├── examples/│   ├── mcp/│   ├── curl/│   ├── python/│   ├── javascript/│   ├── openai-agents/│   └── microsoft-agent-framework/│└── scripts/    └── apify/

Revenue initiative

The repository contains automation for:

  • testing and deployment
  • Apify runtime configuration
  • pay-per-event pricing
  • Store / agentic-payment eligibility
  • external paid-event telemetry
  • milestone issues at 1, 10, and 100 external paid events

See:

text
AUTONOMOUS_REVENUE_INITIATIVE.md

Security principles

Industrial Platform services should:

  • block private/local-network fetch targets when accepting arbitrary URLs
  • bound response sizes, timeouts, and concurrency
  • use structured machine-readable outputs
  • avoid silently fabricating unavailable data
  • charge successful results rather than failed fetches where practical
  • keep credentials in secret stores rather than source code

License

Integration examples and open-source code in this repository are provided under the MIT License unless otherwise noted.

Industrial Platform

Organization:

text
industrial-platform-ai

Apify namespace:

text
industrial_platform

Machine-readable catalog:

text
catalog.json

来源：README.md，提交 dcb23c4

工具

0
工具元数据尚未被收录。

版本历史

1
  1. v1.0.2最新Sep 30, 2026