Wet Mcp

WET - Web Extended Toolkit MCP Server

Distribution & CLI: the repo, CLI and Python module are wet. The PyPI distribution is n24q02m-wet (install pip install n24q02m-wet / uvx n24q02m-wet). wet is an HTTP-only MCP server: start it with wet server start and point clients at http://127.0.0.1:8000/mcp. There is no stdio mode; a bare wet prints the control-plane usage. The wet and wet-mcp aliases remain available.

mcp-name: io.github.n24q02m/wet

Open-source MCP server for AI agents: web search, content extraction, and library docs.

Current release: v3.x. media(action="analyze") was removed in the v2.0.0 BREAKING release. Use imagine-mcp's understand action for vision/audio/video analysis. See docs/migration.md for the upgrade recipe.

[CI] [codecov] [PyPI] [License: Apache-2.0]

[Python] [SearXNG] [MCP] [semantic-release] [Renovate]

Table of contents

• Features
• Status
• Quick install
• Self-host usage
• Configuration
• Documentation
• Tools
• CLI
• Comparison
• Security
• Build from Source
• Trust Model
• License

Features

• Web Search -- Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache (1 h general / 5 min time-sensitive), standardized citation format, and 200-token snippet cap. Optional cloud search backends (Tavily, Brave, Exa, Kagi, OpenRouter, Firecrawl) as a fallback chain via SEARCH_BACKENDS
• Academic Research -- Search Google Scholar, Semantic Scholar, arXiv, PubMed, CrossRef, BASE
• Library Docs -- Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs
• Content Extract -- 5-strategy escalation chain via n24q02m-web-core ScrapingAgent (basic_http -> tls_spoof -> render backends from BROWSER_BACKENDS (native / browserless / cf-browser-rendering) -> optional key-gated captcha), markitdown bridge for low-tier HTML/MD fallback, smart chunks structured output (clean text + markdown + JSON-LD + code blocks + metadata), batch processing (up to 50 URLs), deep crawling, site mapping
• Local File Conversion -- Convert PDF, DOCX, XLSX, CSV, HTML, EPUB, PPTX to Markdown
• Media -- List + download images / videos / audio files. analyze was removed in v2.0.0 -- use imagine-mcp.understand for vision/audio inference
• Anti-bot -- Stealth strategies bypass Cloudflare, Medium, LinkedIn, Twitter
• Zero Config -- Built-in local reference embedding + reranking through fastretrieval, no API keys needed. Optional cloud models configured per task via the [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml (OpenRouter default; one OPENROUTER_API_KEY serves every cell)

Quick install

# Method 1 (default): plugin install via Claude Code/plugin marketplace add n24q02m/claude-plugins/plugin install wet@n24q02m-plugins
# Method 2 (CLI): start a local instance, then register its HTTP endpointuvx --python 3.13 --from n24q02m-wet wet server startclaude mcp add --transport http wet http://127.0.0.1:8000/mcp
# Method 3 (always-on): compose stack from a clone of this repo (token auth)docker compose up --build -d
# Method 4 (remote): point a client at an HTTP deploymentclaude mcp add --transport http wet https://<your-host>/mcp

Every method needs a running instance: wet never runs as a client-spawned subprocess. Install matrix (all Streamable HTTP; see the Setup page for full steps):

Public OCI image publication is discontinued. Existing historical registry tags remain untouched; new container deployments build from source or use the Cloudflare-managed registry.

The HTTP endpoint speaks Streamable HTTP. With auth = "token" or "multi" the client sends Authorization: Bearer <token>; no-auth is loopback-only. Stand one up via Method 2 or 3.

Full setup matrices live at the canonical docs site mcp.n24q02m.com/servers/wet/setup/ and the paste-to-agent snippets at claude-plugins/plugins/wet/setup-with-agent.md (per Spec F single source of truth).

Self-host usage

Two supported ways to run the always-listening HTTP server. In both, the MCP endpoint is http://127.0.0.1:8000/mcp (Streamable HTTP) and the config root is ~/.wet/ (config.toml, docs.db, subs/). Never run the server as a spawned subprocess of a client — register the endpoint in your client instead.

Dev (uv, no-auth loopback)

git clone https://github.com/n24q02m/wet && cd wetuv run wet config init            # writes ~/.wet/config.toml (default: auth = "no-auth")uv run wet server start           # serves http://127.0.0.1:8000/mcp (--foreground to stay attached)

no-auth refuses non-loopback binds — the server is localhost-only until you switch to token auth.

Always-on (docker)

# 1. Mint a token and its scrypt hash (hull-core canonical; the hash command#    never echoes the token itself)openssl rand -hex 32                 # the token — give it to clients, keep it secretuv run wet token hash <token>        # paste the output as token_hash
# 2. Instance config from the examplecp docker-config/config.example.toml docker-config/config.toml#    edit [server] token_hash and the [models.*] cells (see below)
# 3. Data continuity (optional): carry an existing instance over by copying#    docs.db and subs/ into the wet-home named volume BEFORE first start:docker volume create wet-wet-home && \  docker cp ~/.wet/docs.db wet-wet-home:/docs.db && \  docker cp ~/.wet/subs wet-wet-home:/subs  # adjust: files land as root, chown 999:999
docker compose up -d --build

Compose publishes 127.0.0.1:${WET_PORT:-8000} → container 8000 (loopback only) and mounts docker-config/config.toml read-only at /home/appuser/.wet/config.toml. Persistent data (docs.db, subs/) lives in the wet-home named volume; the download/cache dir in wet-data. Editing the config takes effect on docker compose restart.

Consumers

# CLI (talks to the running server over HTTP; exits 2 with a hint when none runs):uvx --python 3.13 --from n24q02m-wet wet search "<query>"
# MCP client (Claude Code):claude mcp add --transport http wet http://127.0.0.1:8000/mcp \  --header "Authorization: Bearer <token>"
# Raw probe:curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \  -X POST -H 'Content-Type: application/json' \  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'            # 401curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \  -X POST -H 'Authorization: Bearer <token>' -H 'Content-Type: application/json' \  -H 'Accept: application/json, text/event-stream' \  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}'  # 200

Config: local vs cloud models

The [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml are per-task (each its own base_url + api_key + model, OpenAI-spec). The example ships OpenRouter examples; any OpenAI-compatible endpoint works — cloud, or a local server such as Ollama (base_url = "http://host.docker.internal:11434/v1" from the container). Keys in the file are host-only material; alternatively leave api_key = "" and inject at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY.

Configuration

wet runs zero-config out of the box: web search uses an embedded local SearXNG, and embedding/reranking fall back to the bundled local ONNX models through fastretrieval when no cloud keys are set. For higher-quality results, configure the per-task provider cells in ~/.wet/config.toml. Operational settings are plain environment variables (no app prefix) -- in HTTP self-host mode configure them via environment variables or ~/.wet/config.toml.

Provider cells -- [models.embed|rerank|chat|jev_score] each take their own base_url + api_key + model (OpenAI-spec). OpenRouter is the pre-wired default for every cell; any OpenAI-compatible endpoint works. Cell keys may be left empty in the file and injected at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY. When a cell keeps the OpenRouter default base_url, OPENROUTER_API_KEY in the environment serves it as a last-resort credential — one OpenRouter key can power every task. Empty embed/rerank cells fall back to the local ONNX models; an unkeyed chat cell disables LLM features (e.g. extract(action="agent")).

FASTRETRIEVAL_CACHE_PATH controls the local model cache.

Search backends -- SEARCH_BACKENDS is an ordered runtime fallback chain: searxng (default, local or external via SEARXNG_URL), keyed tavily / brave / exa / kagi / openrouter, optional-key firecrawl, and credential-free duckduckgo / startpage. Keyed providers use TAVILY_API_KEY, BRAVE_API_KEY, EXA_API_KEY, KAGI_API_KEY, or OPENROUTER_API_KEY (comma-separate for rotation). The openrouter backend runs the query through a chat completion with the openrouter:web_search server tool and maps the returned url_citation annotations onto the shared result shape; OPENROUTER_MODEL (default a free-tier model), OPENROUTER_BASE_URL, and OPENROUTER_SEARCH_ENGINE override its behavior. Firecrawl attempts a keyless request when FIRECRAWL_API_KEY is absent; rejection or a DuckDuckGo/Startpage bot challenge advances the chain.

Result reranking uses the configured rerank cell ([models.rerank] — cloud via OpenRouter by default, or local ONNX when unkeyed); the same chain serves web search, research, similar-page search, agent search, and docs discovery/indexing fallbacks. SearXNG retains its science-category filter for research; other providers use their own search capabilities.

Browser render backends -- BROWSER_BACKENDS (CSV, escalation chain) picks the headless render leg of extract: native (in-process chromium, the zero-config default), browserless (self-host render service -- set BROWSERLESS_URL + BROWSERLESS_TOKEN), and cf-browser-rendering (Cloudflare Browser Rendering -- set CF_ACCOUNT_ID + CF_BROWSER_RENDERING_TOKEN). Empty chain falls back to native. Set CAPSOLVER_API_KEY to append an optional, key-gated CAPTCHA tier as the last escalation step.

Robots policy -- set RESPECT_ROBOTS_TXT=true to enforce robots.txt across both the extract strategy chain and the Crawl4AI-backed crawl, sitemap, and list_media actions. The default is false to preserve existing deployment behaviour; configure this process-level policy explicitly when the operator requires robots enforcement.

Invisible tier + seeded identity (opt-in) -- append invisible to BROWSER_BACKENDS to add a stealth-Firefox engine tier as the last escalation step (n24q02m-hull[invisible] extra: pip install "n24q02m-wet[invisible]"). One coherent browser identity spans the whole chain: seed it with WET_IDENTITY_SEED (or let wet derive once and persist it under ~/.wet/subs/<sub>/identity.json) and install the wet[identity] extra. Without the extras the chain keeps legacy behaviour (warn-once). STEALTHFOX_BINARY points at a patched Firefox build to skip the engine download; IDENTITY_PROFILE_DIR (default ~/.wet/subs/<sub>/profiles) keeps persistent browser profiles per namespace.

Disable local fallbacks -- opt out of the heavy in-process local fallbacks per capability (e.g. on a slim container that renders/searches/embeds via cloud backends only): DISABLE_LOCAL_BROWSER, DISABLE_LOCAL_SEARCH, DISABLE_LOCAL_EMBED, DISABLE_LOCAL_RERANK.

Auth -- [server] auth in ~/.wet/config.toml is no-auth (loopback only), token (store only token_hash, see wet token hash) or multi (per-user tokens). Server-host settings such as OPENROUTER_API_KEY belong to the process that runs wet server start (or the compose environment), not to the MCP client config.

Example client config (add headers with the Bearer token for token / multi):

{  "mcpServers": {    "wet": {      "type": "http",      "url": "http://127.0.0.1:8000/mcp"    }  }}

To use a different endpoint or model per task, edit the [models.*] cells in ~/.wet/config.toml, or inject HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY at start instead of storing keys in the file.

Status

Stable architecture with a single transport: Streamable HTTP, always served by wet server start (self-host). No stdio mode, no daemon-bridge layer and no auto-spawn by clients. The media.analyze action was removed in the v2.0.0 BREAKING release -- see docs/migration.md for the upgrade recipe. Current release line: v3.x.

Documentation

Full docs at mcp.n24q02m.com/servers/wet/setup/:

• Setup -- install methods for Claude Code, Codex, Gemini CLI, Cursor, Windsurf, mcp.json
• Modes overview -- transports and HTTP auth modes
• Multi-user setup -- per-JWT-sub credential model

In-repo references (Spec F single source of truth: setup docs live in claude-plugins/plugins/wet/):

• docs/ARCHITECTURE.md -- web-core ScrapingAgent integration, strategy chain, storage layout, LLM provider dispatch
• docs/BENCHMARKS.md -- v1.x baseline coverage / latency placeholders + tier-1 fixture metrics

Install with AI agent -- paste this to your AI coding agent:

Install MCP server wet following the steps at https://raw.githubusercontent.com/n24q02m/claude-plugins/main/plugins/wet/setup-with-agent.md

Tools

5 MCP tools (3 domain + config + help). The legacy setup tool merged into config action dispatch.

Media boundary: For vision / audio understanding (image captioning, OCR, audio transcription, video summarization), use imagine-mcp. media.analyze was removed in wet v2.0.0 -- use imagine-mcp.understand instead.

CLI

The package installs three console scripts: n24q02m-wet (primary package command), wet (short alias), and wet-mcp (legacy alias). wet is the control plane for the HTTP server plus one thin client; a bare invocation prints its usage.

uvx --from n24q02m-wet wet warmup   # try a subcommand without a persistent install
wet server start                # start the Streamable HTTP server (detached; --foreground to attach)wet server status               # pid, lock and HTTP livenesswet server stop
wet warmup                      # pre-download local models + run auto-setup (SearXNG, browser) to avoid first-run delayswet docs reindex <library>      # drop the cached docs index for <library>; the next docs search re-indexes it

auth google accepts an optional bring-your-own OAuth client via --client-id and --client-secret (single-user / local machine only; the token is written to the local store). Each subcommand prints a JSON result and exits.

Security

• SSRF prevention -- URL validation on crawl targets
• Graceful fallbacks -- Cloud → Local embedding, multi-tier crawling
• Error sanitization -- No credentials in error messages
• File conversion sandboxing -- Defaults to the working directory and temp dir; override with CONVERT_ALLOWED_DIRS

Build from Source

git clone https://github.com/n24q02m/wet.gitcd wetuv syncuv run wet

Trust Model

This plugin implements TC-Local (machine-bound, single trust principal). See mcp-core trust model for full classification.

License

Apache-2.0 -- See LICENSE.