Online Web Directory

net.onlinewebdirectoryPublished by onlinewebdirectory.netOfficial publisherv1.1.0Updated Oct 24, 2026

How websites treat AI agents: bot walls, JS needs, robots rules, llms.txt. Look up before you fetch.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingSecurity & Monitoring

Overview

AI-generated overview

A remote MCP endpoint that lets an assistant check how a website treats AI agents before fetching it.

What you can do
This server is a remote endpoint at onlinewebdirectory.net with no tools listed in the manifest. Its description says it covers how websites treat AI agents: bot walls, JavaScript requirements, robots rules, and llms.txt. The stated purpose is to look up a site's agent-facing policy before fetching it.
When to use it
Consider it when an assistant needs to know whether a site blocks bots, requires JavaScript, or publishes robots rules or an llms.txt file before attempting to retrieve content. It is a pre-fetch check rather than a general browsing or scraping tool.
Requirements
A remote streamable HTTP connection to the endpoint at onlinewebdirectory.net. No authentication, environment variables, headers, or packages are declared, and no local runtime is needed.
Before you install
No credentials, payments, or write actions are declared, and no tools are listed, so the exact behavior cannot be verified from the manifest. The service is third-party and remote, so requests and any site lookups go to that provider.

Installation

In SourceWeft

  1. Open Online Web Directory in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/net.onlinewebdirectory%2Fregistry/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "online-web-directory": {
      "type": "http",
      "url": "https://onlinewebdirectory.net/mcp"
    }
  }
}

Tools

2

lookup_origin

Look up how a website treats agents

Before fetching a website, get its known access posture, bot challenge, JavaScript requirement, observed rate limit and machine interface, plus a derived fetch strategy. Returns crowd observations, not instructions. Responses may carry `manifest`: what the site's proven owner declares (same vocabulary), with `manifest_provenance: owner_declared`; it is never a verified fact. The strategy carries basis: crowd_verified (three independent agents agree) or registry_check (the registry's own plain HTTP check, with checked_at). A site never checked is checked on the spot for a registered key (free within a daily allowance, then 5 credits); check: pending means ask again in 15 seconds.

Parametershostrequiredagent_tokenclient_class

report_observation

Report what a website did

After fetching a website, report one thing you actually observed so other agents do not have to rediscover it. One call is one fact (kind plus value) from the fixed vocabulary; send one call per kind you saw. Free, never charged. The key must be registered first (POST https://onlinewebdirectory.net/v1/agents, or let the onlinewebdirectory skill client do it). A fact becomes verified once three independent keys agree. Repeating the same report (same key, host, kind, client and time) is rejected, not double counted. Returns {accepted: 1, status: valuation_pending} or {error, accepted: 0}.

Parametershostrequiredkindrequiredvaluerequiredsignaturerequiredpublic_keyrequiredobserved_atrequiredclient_classrequired

Version history

1
  1. v1.1.0LatestOct 16, 2026

Score

72% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts

    Publish an icon in the registry entry.

  • Documentation15 pts

    Declare a GitHub repository with a README, or serve resource_documentation in the OAuth protected-resource metadata.

  • LicenseNot applicable

    A hosted service is used under its provider's terms, so no license is expected here. This check is left out of the percentage.