Sitemap URL Extractor for AI Agents
io.github.industrial-platform-aiPublished by GitHub account industrial-platform-aiv1.0.2Updated Oct 25, 2026
x402 sitemap discovery and URL extraction for agents. $0.0001 per emitted URL.
Overview
A hosted, pay-per-use MCP service that discovers sitemaps and expands them into clean URL rows for AI agents.
- What you can do
- Discover and expand public sitemaps into clean URL rows, billed per emitted URL at $0.0001 each, through the sitemap-intelligence service on the hosted Apify MCP endpoint. Tool discovery is free; paid calls run through x402 or Apify authentication. The same endpoint can also expose related paid services such as web metadata, web change, article text, PDF text and research briefs.
- When to use it
- Use it when an agent needs a list of URLs from a site's sitemap for crawling, indexing or SEO workflows, and delegating that extraction is simpler than building it. It is a paid service, so it suits workflows that can settle small per-URL charges.
- Requirements
- A remote MCP client connecting to the hosted Apify endpoint, with the sitemap-intelligence tool selected. Paid execution needs a supported payment or authentication path: a funded Base wallet for x402 USDC settlement, or Apify OAuth or an Apify API token. Network access to the endpoint is required.
Installation
In SourceWeft
- Open Sitemap URL Extractor for AI Agents in the dashboard and add it to a workspace.
- Enable the server for the chats that should use its tools.
Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.
Use with an agent
Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.
Read https://sourceweft.com/mcp/io.github.industrial-platform-ai%2Fsitemap/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.Other MCP clients
Add this to your client's mcpServers config.
{
"mcpServers": {
"sitemap-url-extractor-for-ai-agents": {
"type": "http",
"url": "https://mcp.apify.com/?tools=industrial_platform/sitemap-intelligence&payment=x402"
}
}
}README
Industrial Platform Agent Tools
Network trust governance
Industrial Platform uses the Industrial Platform Trust Protocol (IPTP) to separate Base USDC settlement, optional wallet-control verification, independent commercial attribution, and demand-gated factory activation. Identity verification is never a prerequisite to an ordinary x402 purchase. Current implementation is partial; see the protocol for evidence and enforcement boundaries.
Direct recurring x402 MCP
For recurring autonomous workloads, use the direct x402-native MCP first:
Literal recurring tool names are optimized for auto-import and delegation:
monitor_wallet, transaction_status, watch_transaction, treasury_snapshot, pretrade_context, crypto_candles, crypto_price, crypto_book, market_snapshot, and monitor_webpage_change.
These map to the same canonical paid HTTP workloads, so importer compatibility increases without splitting HTTP settlement history across duplicate routes.
Open-source integration toolkit for connecting AI agents, MCP clients, autonomous workflows, agent orchestrators, and developer applications to Industrial Platform machine-callable services.
Industrial Platform provides specialized paid utilities that other agents can discover and invoke instead of rebuilding research, monitoring, extraction, and document-processing capabilities inside every workflow.
Install in your agent IDE
For the lowest-friction recurring workflow, install Web Metadata + Web Change first. Both currently complete in about 3 seconds median and cost $0.01 per successful unit.
- Cursor: one-click install
- VS Code / GitHub Copilot: one-click install
- GitHub Copilot CLI:
copilot mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402' - Claude Code:
claude mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402' - Cline:
cline mcp install industrial-platform-fast-web --transport http 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'
See INSTALL.md for the full installer matrix, recurring-use patterns, and the full portfolio endpoint.
One MCP endpoint for the full portfolio
This hosted MCP endpoint exposes the six currently runnable Industrial Platform services through one direct x402 integration. Tool discovery is free; autonomous callers pay only when invoking a paid Actor.
Current services
Live pricing and execution eligibility should be verified against Apify Store before paid execution.
Official MCP Registry
Industrial Platform is published in the official MCP Registry as:
Registry publication is automated on merge.
The Registry listing points to the same hosted six-service Apify MCP bundle used by this repository.
Individual MCP Registry entries
For search-oriented discovery, each paid service also has its own active MCP Registry entry:
These individual entries point to the same Apify-hosted Actor tools as the portfolio bundle and improve discovery for capability-specific MCP searches.
Machine-readable discovery
The repository publishes:
Service catalog
catalog.json is the portfolio-level machine-readable catalog. It contains Actor IDs, Store URLs, billing events, current prices, and common delegation targets.
MCP bundle manifest
manifests/mcp-bundle.json defines the hosted MCP endpoint and the Actors included in the bundle.
MCP client configuration
examples/mcp/remote-server.json can be used as a starting point for MCP clients that accept remote server configuration.
Direct x402 framework integrations
Industrial Platform also exposes a direct Base x402 gateway for autonomous callers that do not need the Apify MCP layer:
Machine-readable discovery:
Direct x402-native MCP:
The MCP server exposes the same paid tool catalog over Streamable HTTP and performs x402 negotiation at tool-call time. Tool discovery is free; a caller wallet signs the advertised Base USDC requirement when it invokes a paid tool.
Ready-to-copy adapters and workflows are included for:
- Minimal official x402 caller — official x402 client, automatic 402 → payment → retry, with a hard per-payment ceiling
- Coinbase AgentKit — Base wallet + AgentKit's native x402 provider, including a dry-run-by-default 1-atomic-USDC clean-room canary
- ElizaOS — native @elizaos/plugin-wallet x402 client with explicit budget controls and payment disabled by default
- LangChain / LangGraph
- CrewAI
- LlamaIndex
- SEO crawler workflow
- Crypto intelligence workflow
The examples use the caller's own funded Base wallet, negotiate x402 automatically, and preserve framework-level tool choice rather than forcing Industrial Platform calls.
Agent-to-agent paid quick start
Fund the generated wallet with USDC on Base before paid calls. Product #7 (Link Intelligence) is not included here until its public/agentic publication is confirmed.
Quick start
MCP
Configure one remote MCP server:
Compatible clients can then discover the exposed Actor tools from that server.
REST / Apify API
Every service can also be invoked directly through Apify's Actor API.
General Actor identifier form:
Example:
Do not commit API tokens, wallet credentials, secrets, or private keys to this repository.
When should an agent delegate?
A calling agent should prefer a specialized Industrial Platform service when delegation is cheaper, simpler, or operationally safer than rebuilding the capability internally.
Examples:
Portfolio architecture
Execution and monetization currently run through Apify.
Authentication and machine payments
Paid execution requires a supported authenticated or agentic-payment path.
Depending on the integration, this may use:
- Apify OAuth
- Apify API tokens
- Apify-supported agentic payment infrastructure
Store eligibility and pricing can change. Calling systems should inspect current Actor metadata before executing paid services.
Repository structure
Revenue initiative
The repository contains automation for:
- testing and deployment
- Apify runtime configuration
- pay-per-event pricing
- Store / agentic-payment eligibility
- external paid-event telemetry
- milestone issues at 1, 10, and 100 external paid events
See:
Security principles
Industrial Platform services should:
- block private/local-network fetch targets when accepting arbitrary URLs
- bound response sizes, timeouts, and concurrency
- use structured machine-readable outputs
- avoid silently fabricating unavailable data
- charge successful results rather than failed fetches where practical
- keep credentials in secret stores rather than source code
License
Integration examples and open-source code in this repository are provided under the MIT License unless otherwise noted.
Industrial Platform
Organization:
Apify namespace:
Machine-readable catalog:
Source: README.md at commit 1a3a07f
Tools
0Version history
1- v1.0.2LatestSep 30, 2026
Score
85% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
- Reached in the last 30 daysRequired25 pts
- Auth declaredRequired10 pts
- Tools listedRequired15 pts
Answer tools/list without a credential, or ship a package that starts in a sandbox without one.
- DescriptionRequired15 pts
- Logo10 pts
- Documentation15 pts
- License10 pts
