
Argus Retrieval
io.github.Khamel83Published by GitHub account Khamel83v1.6.5Updated Oct 24, 2026
Multi-provider search broker for AI agents: 14 providers, 12-step extraction, retrieval workflows.
The publisher's own server: Argus: Belgian tender insights
Overview
Argus Retrieval is a local MCP adapter that gives an assistant multi-provider web search, page extraction, dead-link recovery and research-pack workflows.
- What you can do
- Search the web across many providers with discovery, research, recovery and grounding modes, optional free-only routing and per-provider score attribution
- Extract clean page text from a URL and expand links found on a page
- Recover dead or moved URLs and articles through archive fallbacks
- Capture a site's important pages and build local docs-plus-research packs
- Check search health and provider budgets, and keep multi-turn context via session IDs
- When to use it
- Useful when an assistant needs reliable web retrieval with fallbacks rather than a single search API: research and fact-checking, reading full page content, rescuing broken links, or assembling local evidence packs for RAG work.
- Requirements
- Runs as a local Python process (Python 3.11+, installed via uvx or pipx from the argus-search package). The MCP adapter normally delegates to an authenticated HTTP authority, needing ARGUS_AUTHORITY_URL and ARGUS_AUTHORITY_TOKEN; a development-only standalone mode must be explicitly enabled. Optional provider keys such as ARGUS_BRAVE_API_KEY, ARGUS_TAVILY_API_KEY, ARGUS_SERPER_API_KEY, ARGUS_EXA_API_KEY, ARGUS_LINKUP_API_KEY and ARGUS_WOLFRAM_API_KEY unlock extra providers; SearXNG can be…
Installation
In SourceWeft
- Open Argus Retrieval in the dashboard and add it to a workspace.
- Enable the server for the chats that should use its tools.
Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.
Other MCP clients
Follow the launch instructions in the repository.
README
Argus
[Python 3.11+] [PyPI Version] [PyPI Downloads] [License: MIT] [CI] [MCP Registry] [Docker]
Retrieval platform for AI agents. Argus routes search across 14 providers, recovers dead URLs, captures important site content, builds local docs-plus-research packs, and persists everything with traceable local artifacts.
Features at a glance:
- Topology-aware acquisition — Argus knows if it's on a residential IP or datacenter, routing search and extraction automatically to avoid blocks and minimize network hops.
- 14 providers, one API — free-first tier routing, budget-exhausted providers skipped automatically
- Zero-key start —
pip install argus-searchgives you DuckDuckGo + Yahoo immediately, no accounts needed - SearXNG self-host = 70+ engines — Google, Bing, Yahoo, Startpage, Ecosia, Qwant and more via one Docker container
- 12-step content extraction — returns full page text with quality gates, not just links
- Opinionated retrieval workflows — recover dead articles, capture important pages from a site, and build local docs-plus-research packs
- Argus-owned corpus storage — runtime data goes to a writable user data directory, not your repo checkout
- Multi-turn sessions — pass
session_idfor conversational context across searches - Score attribution — optionally show which providers contributed to each fused RRF score
- Usage dashboard — inspect provider budgets, recent query volume, and machine-level usage at
/dashboard - 4 search modes — discovery, research, recovery, grounding
- Dead URL recovery —
/recover-urlwith Wayback Machine and archive fallbacks - 4 integration paths — HTTP API, CLI, MCP server, Python SDK
Built for AI agent builders, RAG pipelines, and ops teams who need reliable search, capture, and local evidence without stitching APIs together.
Status: beta. The retrieval workflows and corpus model are production-oriented, but still maturing.
Status: see the public status page. Authorized maintainers can use the private argus-ops README for the latest dated reports.
Current production checkpoint — September 7, 2026: The authenticated HTTP API and MCP adapter are operational, and sampled free search works through SearXNG, Yahoo, and GitHub. DuckDuckGo is intermittent and fails closed after acquisition-policy blocks. Production readiness is
ready=true, degraded; paid providers are not currently admitted until their non-secret credential-version and account-scope bindings are recorded. Browser capability and recovery metadata evidence remain open. See the current status matrix before relying on a provider.
Contents
- Quickstart
- Development
- Where Argus Writes Data
- Opinionated Workflows
- Providers
- HTTP API
- Dashboard
- Integration
- Content Extraction
- Architecture
- Configuration
- When Not To Use Argus
- FAQ
Quickstart
Mode 1: Local CLI (zero config)
That's it. DuckDuckGo handles the search — no accounts, no keys, no containers. You get unlimited free search from your laptop right now. Add API keys whenever you want more providers, or don't.
Works on any machine with Python 3.11+ — laptop, Mac Mini, Raspberry Pi, cloud VM. Nothing to host.
For MCP (Claude Code, Codex, OpenCode, Cursor, VS Code):
That writes native config for Claude Code, Codex CLI, OpenCode, and Cursor. Restart the client after configuration. For manual stdio setup:
Or install from the MCP Registry:
Standalone development needs no server or keys, but it must be explicitly enabled. Production MCP always delegates to an authenticated HTTP authority.
See MCP Client Setup for exact config files, verification commands, remote HTTP setup, and troubleshooting.
Mode 2: Full Stack Server
Got a Raspberry Pi running Pi-hole? A Mac Mini on your desk? An old laptop? That's enough to run the full stack — SearXNG (your own private search engine, disabled by default) plus local JS-rendering content extraction.
SearXNG takes 512MB of RAM and gives you a private Google-style search engine (disabled by default — set ARGUS_SEARXNG_ENABLED=true) that nobody can rate-limit, block, or charge for. It runs alongside Pi-hole on hardware millions of people already own.
Where Argus Writes Data
Argus code and Argus runtime data are different things.
- Code lives wherever you install or clone Argus.
- Runtime corpus data lives in a writable user data directory resolved by
platformdirs, or inARGUS_DATA_ROOTif you override it.
Inspect the exact paths on your machine:
By default Argus writes:
- official docs cache under the resolved
docs/cache/ - research packs under
docs/research/ - workflow run state under
workflows/runs/ - versioned workflow snapshots under
snapshots/
This means Argus does not require a sibling ../docs-cache checkout. If you have an older docs-cache tree, import it once with:
Opinionated Workflows
These workflows build local artifacts, not just transient JSON responses.
Recover A Dead Article
Argus searches for recovery candidates, extracts the best result, saves the recovered sources locally, and writes a citation-backed report plus manifest.
Capture The Important Parts Of A Site
Argus stays on-domain, uses sitemap-assisted discovery plus heuristic link scoring, saves the important pages it finds, and writes a detailed summary with references.
Build A Docs + Research Pack
Argus captures official docs into its local docs cache, adds non-official supporting sources from search, and writes a combined research pack with traceable artifacts.
Development
Repo development is pinned to Python 3.12. The package runtime floor is Python
3.11, the production image runs Python 3.12.3, and Python 3.13 is the
compatibility CI lane. Required CI passes all three; contributors should use
the uv workflow below so local verification matches the canonical lane and
does not accidentally use an older system interpreter.
The repo includes .python-version with 3.12 so uv, pyenv, and similar tools pick the right interpreter by default. More contributor guidance lives in CONTRIBUTING.md.
Providers
¹ SearXNG aggregates Google, Bing, Yahoo, Startpage, Ecosia, Qwant, Wikipedia, and 60+ more — all behind a single self-hosted endpoint. Run docker compose up -d on any machine with 512MB of free RAM.
² WolframAlpha returns computed answers (math, unit conversions, factual lookups), not web search results. It only activates in grounding and research modes. Queries it can't compute (general web searches) return empty — no error, no health penalty.
7,000+ free queries/month from recurring free-tier providers with API keys (WolframAlpha 2k + Brave 2k + Tavily 1k + Exa 1k + Linkup 1k), or up to 12,000+ when Parallel's monthly credit is available to an eligible account with a card on file. DuckDuckGo, Yahoo, and GitHub have no monthly cap. SearXNG is disabled by default (enable in .env). Routing priority: Tier 0 (free: SearXNG*, DuckDuckGo, Yahoo, GitHub, WolframAlpha) → Tier 1 (monthly recurring: Brave, Tavily, Exa, Linkup, Parallel) → Tier 3 (one-time: Serper, You.com, Valyu, SearchAPI). Budget-exhausted providers are skipped automatically.
These are package-level provider tiers and advertised quotas, not proof of current production availability. The production authority fails closed for a credentialed provider until its registration fingerprint, account scope, budget, and approved test evidence are present. A protected key value alone does not prove that the current key works; see docs/STATUS.md for the live classification.
HTTP API
All endpoints prefixed with /api. OpenAPI docs at http://localhost:8000/docs.
Local loopback calls can use the API without auth. Remote HTTP callers must send ARGUS_API_KEY as either Authorization: Bearer ... or X-API-Key: .... Privileged routes under /api/admin/* require ARGUS_ADMIN_API_KEY (or fall back to ARGUS_API_KEY if no separate admin key is configured).
/api/health remains a 200 liveness compatibility route. It intentionally
does not check PostgreSQL, providers, Maya, or the browser, so a dependency
outage cannot cause container restart storms. See
production operations for the canonical topology and
operator procedures, and operational status for endpoint semantics,
readiness classification, observation expiry, and safe telemetry.
Search modes
Tier-based routing always applies first. Within each tier, the mode selects provider order.
Response format
Each result includes url, title, snippet, domain, provider, and score. The traces array shows which providers were called and their outcomes.
When include_attribution is true, each result also includes
score_attribution: a provider-to-score map that decomposes the result's
Reciprocal Rank Fusion score. RRF is additive, so each provider's attribution is
exactly its own rank contribution, and the values sum to score. Attribution is
off by default and cached separately from non-attributed searches.
Budgets
Each provider tracks usage. Tier 1 (monthly) uses a 30-day rolling window; tier 3 (one-time) uses a lifetime counter that never resets. When a provider hits its budget, Argus skips it and moves to the next. Free providers (DuckDuckGo, GitHub) have no limit. SearXNG is free but disabled by default. Set ARGUS_*_MONTHLY_BUDGET_USD to enforce custom limits per provider.
Dashboard
Run the HTTP server and open /dashboard:
The dashboard shows provider budget burn, over-pace and exhausted providers, query volume for the last 30 days, usage by machine, and recent provider activity. Budget cards refresh automatically.
Set ARGUS_ADMIN_API_KEY to require dashboard login. If no admin key is set,
the dashboard is open to anyone who can reach the server, which is suitable only
for trusted local use.
For subpath deployment behind a reverse proxy, set ARGUS_ROOT_PATH to the
external path prefix:
That makes dashboard redirects, links, and HTMX fragment URLs work when the
proxy serves Argus at a path such as https://khamel.com/argus/.
For direct public HTTPS, the repo includes a Caddy profile:
For an existing Authentik/nginx deployment, keep authentication at the proxy
layer and set ARGUS_ROOT_PATH to the public prefix.
Integration
CLI
All commands support --json for structured output.
How sessions work
Pass session_id to any search call. Argus stores each query and extracted URL through the same SQLAlchemy repository used by the retrieval ledger (ARGUS_DB_URL, PostgreSQL in production and SQLite for direct local use). Reusing the same session_id gives the broker context from prior queries — follow-up searches are automatically refined using earlier conversation context. Sessions persist across restarts. Omit session_id for stateless, one-shot searches.
Legacy sessions from the former budget SQLite database can be reconciled without mutating the target first:
The import is idempotent: an identical existing session is skipped and a different session with the same ID is reported as conflicting.
MCP
MCP is a stateless execution adapter over the authenticated HTTP API. It does not construct providers or a broker and does not own browser, database, budget, session, health, or outbox state. Configure the adapter process with:
The deployed production endpoint supports both the verified MCP 2025-11-25
compatibility contract and the MCP 2026-07-28 stateless transport revision.
The newer path is one-shot and does not require an initialize handshake or
Mcp-Session-Id; durable policy, budgets, sessions, and evidence remain owned
by the HTTP authority. See
docs/research/2026-08-11-mcp-stateless-production-authority.md
for the source-backed boundary and required no-spend probes.
Option A — Local adapter (stdio)
Install the adapter on the same machine as your MCP client:
Use the full path if argus isn't on PATH: "/home/you/.local/bin/argus".
The adapter inherits ARGUS_AUTHORITY_URL and ARGUS_AUTHORITY_TOKEN from
the client process. To run a local broker instead, development environments
must explicitly set ARGUS_MCP_STANDALONE=true; production rejects it.
Works with Claude Code, Codex CLI, OpenCode, Cursor, and any stdio-based MCP client. Use argus mcp init --global --client all to write native client configs for the current machine.
Detailed client setup and verification commands live in docs/mcp-clients.md.
Option B — Remote MCP adapter (clients over Tailscale)
Run Argus on one machine, connect every client over the network. No local install on clients.
On the adapter host:
Remote MCP credentials must also be valid scoped credentials at the HTTP
authority because the adapter forwards each authenticated bearer token
unchanged. For stdio, ARGUS_AUTHORITY_TOKEN is the caller credential.
To keep the HTTP API and remote MCP service running after reboot on a systemd host:
The installer validates the minimal adapter environment, installs it as
root-only /etc/argus/mcp.env, then installs and starts both units. The MCP
unit never loads the authority's .env, provider vaults, database settings,
browser paths, or writable data volumes.
On each client:
With Tailscale, <server> is your machine's Tailscale IP (e.g. 100.x.x.x). One server, every machine on your mesh gets search.
One-command provisioning:
The script writes Claude/Cursor, Codex, and OpenCode configs on the target.
Local stdio does not require an MCP listener key, but the adapter still
requires its scoped ARGUS_AUTHORITY_TOKEN. Remote MCP mode requires
ARGUS_REMOTE_URL and ARGUS_API_KEY. Requires Python 3.
argus mcp init also generates configs automatically:
Release Status
Pushing main does not publish PyPI. Package and MCP Registry publication happens through the GitHub publish workflow on release creation or manual dispatch. See docs/releasing.md for version sync, preflight checks, and publish verification.
Transports: stdio (default local adapter), sse (legacy remote), and
streamable-http (modern remote, "type":"http" in config). Remote MCP
transports require ARGUS_API_KEY; every transport delegates execution to
ARGUS_AUTHORITY_URL.
Available tools:
- HTTP-backed stdio and remote MCP:
search_web,extract_content,recover_url,expand_links,search_health,search_budgets,recover_dead_article,capture_site, andbuild_research_pack - Explicit standalone development additionally exposes local-only
test_provider,cookie_health,valyu_answer, path/file tools, and resources. These are intentionally absent from production adapters.
search_web accepts free_only=true to restrict results to free (tier-0) providers only, and include_attribution=true to include per-provider score attribution in the Markdown response.
Using Argus from MCP vs HTTP
Two transports, one rule: agents use MCP, everything else uses HTTP.
- MCP (
argus mcp serve) — a stateless authenticated adapter for AI harnesses that speak MCP natively. Core tools delegate to the HTTP authority:search_web,extract_content,recover_url,expand_links, and workflow starts. MCP restarts cannot fork accounting, sessions, health, or outbox state. - HTTP (
POST /api/search,POST /api/extract,POST /api/workflows/...) — for scripts, cron jobs, and service integrations (including Maya). Send a scoped caller credential inAuthorization; body"caller"values are diagnostic labels and cannot override the authenticated identity.
The cross-service transport and role contract for the wider fleet (Maya / Hermes / Argus) is canonical in Maya's architecture documentation.
Python
Direct broker and extraction imports are a standalone development convenience. Production Python callers use the authenticated HTTP API so all execution and durable accounting remain in one authority.
Content Extraction
Argus tries up to twelve methods to extract content from any URL: auth extraction for paywalls, then local extractors (trafilatura, Crawl4AI, Obscura, Playwright, residential IP), then external APIs (Jina, Valyu Contents, Firecrawl, You.com, Wayback, archive.is). Each attempt is quality-checked for completeness and garbage output. See docs/providers.md for the full extractor comparison.
Completeness assessment runs automatically after every successful extraction. Argus scores five signals — trailing ellipsis, feed truncation markers ("Read more", WordPress RSS footers), mid-sentence endings, abrupt final paragraphs, and suspicious round word counts — and returns is_complete, completeness_confidence, and truncation_type alongside the text. When confidence is ≥ 85%, Argus continues trying the next extractor rather than returning a partial result; this means a trafilatura fetch that ends with "..." will automatically fall through to Playwright, Jina, Wayback, etc. Callers that already have text (e.g. RSS feed items) can use POST /api/assess-content to check completeness without triggering extraction.
Obscura (optional) is a lightweight Rust headless browser (~70MB binary, 30MB RAM) with built-in stealth mode — it sets navigator.webdriver=undefined, randomizes canvas/GPU/audio fingerprints per session, and blocks 3,520 tracker domains. This directly addresses bot detection on JS-heavy and anti-scraping sites that block standard Playwright/Chrome. No API key, no rate limit — fully local.
Two ways to use it:
Install the binary: github.com/h4ckf0r0day/obscura/releases
Extract gets the full text of a working URL and tells you whether that text is complete. Recover-URL finds alternatives when a URL is dead, paywalled, or radically changed.
Architecture
Add new providers or extractors with a single adapter file. See CONTRIBUTING.md for the interface.
How a Query Works
-
Cache check.
SearchCachehashes the normalized query, mode, and whether attribution was requested (SHA256). Hit returns immediately with a TTL of 168 hours (7 days). -
Provider queue.
resolve_routing()takes the mode-specific preference list and stable-sorts by tier: tier 0 (free) first, tier 1 (monthly) next, tier 3 (one-time) last. Example for discovery mode: -
Sequential execution with gates. Each provider is checked in order. Four gates must pass before an API call:
- Config — is the provider enabled and configured (API key present)?
- Health — has it failed 5+ consecutive times (triggers 60-minute cooldown)?
- Budget — for tier 1+: is the budget exhausted? For tier 1 (monthly), pacing checks if the 7-day usage rate would drain the remaining budget in under a week — empty days bank headroom. For tier 3 (one-time), a lifetime counter gates access — exhaustion is the sole check.
- Execute — the actual HTTP call. Successes reset failure counters; failures increment them.
-
RRF fusion. Results from all queried providers are merged using Reciprocal Rank Fusion (
k=60). Each result's score is the sum of1/(k + rank)across every provider that returned it. Results appearing in multiple providers rank higher. -
Dedup and truncate. URLs are normalized (stripped
www., tracking params likeutm_*, trailing slashes) and deduplicated. The merged list is truncated tomax_results(default 10). -
Cache and persist. The authority writes the final response to its in-memory cache and configured SQL repository (PostgreSQL in production, SQLite for standalone development). Search results and extractions include provenance metadata (
egress,machine,source_type) for downstream audit. Existing databases are upgraded additively at startup.
Configuration
All config via environment variables. See .env.example for the full list. Limited API-key providers are opt-in: set both the API key and ARGUS_<PROVIDER>_ENABLED=true. Missing keys degrade gracefully — providers are skipped, not errors.
When running from the repo, Argus now auto-loads .env and .env.local (without overriding already-exported environment variables). Disable this behavior with ARGUS_AUTOLOAD_DOTENV=false.
/api/v2/* is additive and returns a canonical version-2 envelope. It remains
fail-closed with unready while the evidence authority is disabled. Unsafe
Host, Origin, credential, media-type, and body-size combinations are rejected
before provider, extractor, session, or persistence work. Version-1 routes
retain their established response shapes.
When Not To Use Argus
Argus is best when you need search, capture, provenance, and local artifacts together.
Avoid it when:
- you only need one search API and do not need fallback or budget controls
- you only need a one-off page scrape with no persistent corpus or report output
- you need an end-user search UI rather than backend retrieval infrastructure
- you need fully deterministic summarization with no heuristic or LLM-assisted steps
FAQ
How is this different from calling Tavily/Serper directly? Argus calls them for you — plus 13 other providers. You get one ranked, deduplicated result set instead of managing multiple API keys and stitching results together. Free providers are tried first, so you only burn credits when needed.
Can I run only one provider? Yes. Set only the API key for the provider you want. All others are silently skipped. For zero-config, just install and go — DuckDuckGo + Yahoo handle search with no keys.
Do I need Docker?
No. pip install argus-search works immediately on any machine with Python 3.11+. Docker is only needed for SearXNG (set ARGUS_SEARXNG_ENABLED=true in .env) or Crawl4AI (local JS rendering).
Which Python version should contributors use?
Use Python 3.12 for repo development and verification: uv sync --python 3.12 --extra dev --extra mcp then uv run pytest tests/ -v --tb=short. The published package still supports Python 3.11+.
What is the safest way to deploy Argus on a network?
Use Tailscale or another private network, bind explicitly to the trusted interface, set ARGUS_API_KEY, and reserve /api/admin/* for ARGUS_ADMIN_API_KEY. Treat direct internet exposure as an advanced mode behind a reverse proxy.
License
MIT — see CHANGELOG.md for release history.
Source: README.md at commit 0e3821e
Tools
0Version history
2- v1.6.5LatestOct 22, 2026
- v1.6.4Sep 16, 2026
Score
50% of the applicable checks met
How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.
- Reached in the last 30 days25 pts
Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.
- Auth declared10 pts
Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.
- Tools listed15 pts
Answer tools/list without a credential, or ship a package that starts in a sandbox without one.
- DescriptionRequired15 pts
- Logo10 pts
- Documentation15 pts
- License10 pts
Other Argus integrations
Argus: Belgian tender insights
ing.argusOfficial publisher
Gives an assistant access to Belgian public tender insights, including bidders, winners, winning prices, and award report scores.

Argus Console — AI Agent Test Lab
Published by GitHub account igor-vii
Argus Console lets an assistant verify agent endpoints, run free simulated or paid real adversarial test batteries against money-moving AI agents, and fetch…

Argus
Published by GitHub account r0ptik
Argus attaches to a live Windows process and gives an AI assistant runtime reverse-engineering evidence: memory bytes, disassembly, imports, call chains and a…

Argus
Published by GitHub account pochtmanr
Lets an assistant drive Argus anti-detect browser profiles, each with its own fingerprint, proxy and cookie jar, through a local API.

Argus
Published by GitHub account ironclawdevs27
Argus runs automated web QA against a locally running app in Chrome and returns severity-ranked bug reports to an AI agent over MCP.