Sitemap URL Extractor for AI Agents

io.github.industrial-platform-ai由 GitHub 帳號 industrial-platform-ai 發布v1.0.2更新於 Oct 25, 2026

x402 sitemap discovery and URL extraction for agents. $0.0001 per emitted URL.

已驗證Streamable HTTP可網頁執行網頁搜尋與抓取商業與電商

概覽

AI 產生的概覽

一個託管的按次付費 MCP 服務,可探索網站地圖並將其展開為乾淨的 URL 清單供 AI 助理使用。

可以做什麼
透過託管 Apify MCP 端點上的 sitemap-intelligence 服務,探索並展開公開網站地圖,輸出乾淨的 URL 資料列,依每筆輸出的 URL 計費 0.0001 美元。工具探索免費;付費呼叫透過 x402 或 Apify 驗證完成。同一端點還可提供網頁中繼資料、網頁變更、文章正文、PDF 文字與研究簡報等相關付費服務。
適用情境
當助理需要從網站地圖取得 URL 清單,用於爬取、索引或 SEO 工作流程,且委派這項擷取比自己實作更簡單時使用。它是付費服務,適合能結算少量按 URL 費用的工作流程。
執行需求
需要一個連接託管 Apify 端點的遠端 MCP 用戶端,並選擇 sitemap-intelligence 工具。付費執行需要受支援的付款或驗證方式:用於 x402 USDC 結算的已儲值 Base 錢包,或 Apify OAuth、Apify API 權杖。需要能存取該端點的網路。
安裝前請注意
呼叫會產生費用:每筆輸出的 URL 0.0001 美元,同一端點上的其他服務更貴(例如每份研究簡報 1.99 美元)。付費呼叫需要已儲值的錢包或 Apify 憑證,呼叫方錢包會在工具呼叫時簽署所宣告的付款要求。價格與資格可能變動,付費執行前應核實目前的 Actor 中繼資料。請勿將 API 權杖、錢包憑證和私鑰提交到程式碼儲存庫。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 Sitemap URL Extractor for AI Agents,將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Web executable,透過 Streamable HTTP。 遠端服務在工作區中設定後即可從網頁執行環境執行。

交給代理程式安裝

把這段提示詞貼上到 Claude Code、Codex、Cursor 等代理程式,或 SourceWeft 對話中。代理程式會讀取該伺服器的安裝指南,先向你展示發布者、端點和登入方式,經你同意後再把它新增到你的客戶端。金鑰不會寫進任何命令。

閱讀 https://sourceweft.com/mcp/io.github.industrial-platform-ai%2Fsitemap/install.md,並按其中說明把這個 MCP 伺服器接入我的客戶端。修改任何設定之前,先告訴我它的發布者、端點和登入方式,等我確認。不要把 API 金鑰寫進命令或我會提交的檔案;問我金鑰應從哪裡取得。

檢視代理程式所依據的安裝指南

其他 MCP 客戶端

把它新增到你客戶端的 mcpServers 設定中。

{
  "mcpServers": {
    "sitemap-url-extractor-for-ai-agents": {
      "type": "http",
      "url": "https://mcp.apify.com/?tools=industrial_platform/sitemap-intelligence&payment=x402"
    }
  }
}

README

Industrial Platform Agent Tools

Network trust governance

Industrial Platform uses the Industrial Platform Trust Protocol (IPTP) to separate Base USDC settlement, optional wallet-control verification, independent commercial attribution, and demand-gated factory activation. Identity verification is never a prerequisite to an ordinary x402 purchase. Current implementation is partial; see the protocol for evidence and enforcement boundaries.

Direct recurring x402 MCP

For recurring autonomous workloads, use the direct x402-native MCP first:

text
https://x402-mcp-gateway-production.up.railway.app/mcp

Literal recurring tool names are optimized for auto-import and delegation:

monitor_wallet, transaction_status, watch_transaction, treasury_snapshot, pretrade_context, crypto_candles, crypto_price, crypto_book, market_snapshot, and monitor_webpage_change.

These map to the same canonical paid HTTP workloads, so importer compatibility increases without splitting HTTP settlement history across duplicate routes.

[AllMCPs Verified]

Open-source integration toolkit for connecting AI agents, MCP clients, autonomous workflows, agent orchestrators, and developer applications to Industrial Platform machine-callable services.

Industrial Platform provides specialized paid utilities that other agents can discover and invoke instead of rebuilding research, monitoring, extraction, and document-processing capabilities inside every workflow.

Install in your agent IDE

For the lowest-friction recurring workflow, install Web Metadata + Web Change first. Both currently complete in about 3 seconds median and cost $0.01 per successful unit.

  • Cursor: one-click install
  • VS Code / GitHub Copilot: one-click install
  • GitHub Copilot CLI: copilot mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'
  • Claude Code: claude mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'
  • Cline: cline mcp install industrial-platform-fast-web --transport http 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'

See INSTALL.md for the full installer matrix, recurring-use patterns, and the full portfolio endpoint.

One MCP endpoint for the full portfolio

text
https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402

This hosted MCP endpoint exposes the six currently runnable Industrial Platform services through one direct x402 integration. Tool discovery is free; autonomous callers pay only when invoking a paid Actor.

Current services

ServiceActorPrimary purposeBilling eventCurrent price
AI Research Briefindustrial_platform/research-brief-agentCurrent, cited multi-source researchresearch-brief-completed$1.99
Web Change Intelligenceindustrial_platform/web-change-intelligenceDetect deterministic changes in public web contentpage-comparison$0.01
Web Metadata Intelligenceindustrial_platform/web-metadata-intelligenceExtract page metadata, OG/Twitter, JSON-LD, headingsmetadata-extraction$0.01
Article Content Intelligenceindustrial_platform/article-content-intelligenceExtract clean article text for RAG/research pipelinesarticle-extraction$0.002
PDF Text Intelligenceindustrial_platform/pdf-text-intelligenceExtract PDF text, per-page text, and metadatapdf-extraction$0.0015
Sitemap Intelligenceindustrial_platform/sitemap-intelligenceDiscover and expand sitemaps into clean URL rowssitemap-url$0.0001

Live pricing and execution eligibility should be verified against Apify Store before paid execution.

Official MCP Registry

Industrial Platform is published in the official MCP Registry as:

text
io.github.industrial-platform-ai/industrial-platform

Registry publication is automated on merge.

The Registry listing points to the same hosted six-service Apify MCP bundle used by this repository.

Individual MCP Registry entries

For search-oriented discovery, each paid service also has its own active MCP Registry entry:

ServiceMCP Registry name
Research Briefio.github.industrial-platform-ai/research-brief
Web Changeio.github.industrial-platform-ai/web-change
Web Metadataio.github.industrial-platform-ai/web-metadata
Article Extractorio.github.industrial-platform-ai/article-extractor
PDF Textio.github.industrial-platform-ai/pdf-text
Sitemapio.github.industrial-platform-ai/sitemap

These individual entries point to the same Apify-hosted Actor tools as the portfolio bundle and improve discovery for capability-specific MCP searches.

Machine-readable discovery

The repository publishes:

text
catalog.jsonmanifests/mcp-bundle.jsonexamples/mcp/remote-server.json

Service catalog

catalog.json is the portfolio-level machine-readable catalog. It contains Actor IDs, Store URLs, billing events, current prices, and common delegation targets.

MCP bundle manifest

manifests/mcp-bundle.json defines the hosted MCP endpoint and the Actors included in the bundle.

MCP client configuration

examples/mcp/remote-server.json can be used as a starting point for MCP clients that accept remote server configuration.

Direct x402 framework integrations

Industrial Platform also exposes a direct Base x402 gateway for autonomous callers that do not need the Apify MCP layer:

text
https://x402-gateway-production-1f21.up.railway.app

Machine-readable discovery:

text
https://x402-gateway-production-1f21.up.railway.app/.well-known/x402https://x402-gateway-production-1f21.up.railway.app/openapi.jsonhttps://x402-gateway-production-1f21.up.railway.app/llms.txt

Direct x402-native MCP:

text
https://x402-mcp-gateway-production.up.railway.app/mcp

The MCP server exposes the same paid tool catalog over Streamable HTTP and performs x402 negotiation at tool-call time. Tool discovery is free; a caller wallet signs the advertised Base USDC requirement when it invokes a paid tool.

Ready-to-copy adapters and workflows are included for:

The examples use the caller's own funded Base wallet, negotiate x402 automatically, and preserve framework-level tool choice rather than forcing Industrial Platform calls.

Agent-to-agent paid quick start

bash
npm install -g @apify/mcpcmcpc x402 initmcpc connect "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402" @industrial-platform --x402mcpc @industrial-platform tools-list

Fund the generated wallet with USDC on Base before paid calls. Product #7 (Link Intelligence) is not included here until its public/agentic publication is confirmed.

Quick start

MCP

Configure one remote MCP server:

json
{  "mcpServers": {    "industrial-platform": {      "url": "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence"    }  }}

Compatible clients can then discover the exposed Actor tools from that server.

REST / Apify API

Every service can also be invoked directly through Apify's Actor API.

General Actor identifier form:

text
industrial_platform~ACTOR-NAME

Example:

text
industrial_platform~pdf-text-intelligence

Do not commit API tokens, wallet credentials, secrets, or private keys to this repository.

When should an agent delegate?

A calling agent should prefer a specialized Industrial Platform service when delegation is cheaper, simpler, or operationally safer than rebuilding the capability internally.

Examples:

text
Need current multi-source research with citations→ research-brief-agent
text
Need to know whether a monitored page changed→ web-change-intelligence
text
Need Open Graph, JSON-LD, canonical, robots, and headings→ web-metadata-intelligence
text
Need clean article text for a RAG or research pipeline→ article-content-intelligence
text
Need text and metadata from a public PDF→ pdf-text-intelligence

Portfolio architecture

text
developer / agent / orchestrator             ↓   Industrial Platform MCP             ↓  specialized machine service             ↓   structured result / event             ↓calling system continues workflow

Execution and monetization currently run through Apify.

Authentication and machine payments

Paid execution requires a supported authenticated or agentic-payment path.

Depending on the integration, this may use:

  • Apify OAuth
  • Apify API tokens
  • Apify-supported agentic payment infrastructure

Store eligibility and pricing can change. Calling systems should inspect current Actor metadata before executing paid services.

Repository structure

text
industrial-platform-agent-tools/│├── README.md├── catalog.json├── AUTONOMOUS_REVENUE_INITIATIVE.md│├── actors/│   ├── web-change-intelligence/│   ├── web-metadata-intelligence/│   ├── article-content-intelligence/│   ├── pdf-text-intelligence/│   ├── sitemap-intelligence/│   └── link-intelligence/│├── packages/│   ├── research-mcp/│   └── change-mcp/│├── manifests/│   └── mcp-bundle.json│├── examples/│   ├── mcp/│   ├── curl/│   ├── python/│   ├── javascript/│   ├── openai-agents/│   └── microsoft-agent-framework/│└── scripts/    └── apify/

Revenue initiative

The repository contains automation for:

  • testing and deployment
  • Apify runtime configuration
  • pay-per-event pricing
  • Store / agentic-payment eligibility
  • external paid-event telemetry
  • milestone issues at 1, 10, and 100 external paid events

See:

text
AUTONOMOUS_REVENUE_INITIATIVE.md

Security principles

Industrial Platform services should:

  • block private/local-network fetch targets when accepting arbitrary URLs
  • bound response sizes, timeouts, and concurrency
  • use structured machine-readable outputs
  • avoid silently fabricating unavailable data
  • charge successful results rather than failed fetches where practical
  • keep credentials in secret stores rather than source code

License

Integration examples and open-source code in this repository are provided under the MIT License unless otherwise noted.

Industrial Platform

Organization:

text
industrial-platform-ai

Apify namespace:

text
industrial_platform

Machine-readable catalog:

text
catalog.json

來源:README.md,提交 1a3a07f

工具

0
SourceWeft 尚未讀取此伺服器的工具。

版本歷史

1
  1. v1.0.2最新Sep 30, 2026

評分

已滿足適用檢查項的 85%

該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通必需25 分
  • 已宣告認證方式必需10 分
  • 已列出工具必需15 分

    請讓伺服器在無憑證時回應 tools/list,或發布可在沙箱中無憑證啟動的軟體包。

  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分