Argus Retrieval

io.github.Khamel83由 GitHub 帳號 Khamel83 發布v1.6.5更新於 Oct 24, 2026

Multi-provider search broker for AI agents: 14 providers, 12-step extraction, retrieval workflows.

發布方自己的服務:Argus: Belgian tender insights

概覽

AI 產生的概覽

Argus Retrieval 是本地 MCP 轉接器,讓助理取得多供應商網頁搜尋、正文擷取、失效連結復原與研究資料包工作流程。

可以做什麼
  • 跨多個供應商搜尋網頁,支援探索、研究、復原與事實查核四種模式,可僅用免費供應商並顯示各供應商的評分歸因
  • 從網址擷取乾淨的正文,並展開頁面中的連結
  • 透過封存回退復原失效或搬移的網址與文章
  • 擷取網站的重要頁面,並建立本機文件加研究資料包
  • 檢視搜尋健康狀態與各供應商預算,並透過工作階段 ID 維持多輪上下文
適用情境
適合需要可靠網頁檢索而非單一搜尋介面的助理情境:研究與事實查核、讀取完整網頁內容、搶救失效連結,或為 RAG 工作整理本機證據資料包。
執行需求
以本機 Python 程序執行(需 Python 3.11 以上,透過 uvx 或 pipx 安裝 argus-search 套件)。MCP 轉接器通常委派給需驗證的 HTTP 服務,需要 ARGUS_AUTHORITY_URL 與 ARGUS_AUTHORITY_TOKEN;僅供開發的獨立模式必須明確開啟。選用供應商金鑰如 ARGUS_BRAVE_API_KEY、ARGUS_TAVILY_API_KEY、ARGUS_SERPER_API_KEY、ARGUS_EXA_API_KEY、ARGUS_LINKUP_API_KEY、ARGUS_WOLFRAM_API_KEY 可啟用更多供應商;也可透過 ARGUS_SEARXNG_BASE_URL 自架 SearXNG。
安裝前請注意
多個選用供應商金鑰屬於機密,部分供應商會消耗有限的免費額度或付費額度,可依供應商設定預算上限。伺服器會在使用者資料目錄或 ARGUS_DATA_ROOT 下寫入本機語料(文件快取、研究資料包、工作流程執行紀錄、快照)。遠端 HTTP 使用需要 ARGUS_API_KEY,管理介面需要 ARGUS_ADMIN_API_KEY。專案處於 beta 階段,部分供應商被回報為降級狀態,結果可能因供應商而異。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 Argus Retrieval,將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Desktop only,透過 STDIO。 STDIO 服務會啟動本機處理程序,因此需要 SourceWeft 桌面主機。

其他 MCP 客戶端

參照 儲存庫 中的啟動說明。

README

Argus

[Python 3.11+] [PyPI Version] [PyPI Downloads] [License: MIT] [CI] [MCP Registry] [Docker]

Retrieval platform for AI agents. Argus routes search across 14 providers, recovers dead URLs, captures important site content, builds local docs-plus-research packs, and persists everything with traceable local artifacts.

Features at a glance:

  • Topology-aware acquisition — Argus knows if it's on a residential IP or datacenter, routing search and extraction automatically to avoid blocks and minimize network hops.
  • 14 providers, one API — free-first tier routing, budget-exhausted providers skipped automatically
  • Zero-key start — pip install argus-search gives you DuckDuckGo + Yahoo immediately, no accounts needed
  • SearXNG self-host = 70+ engines — Google, Bing, Yahoo, Startpage, Ecosia, Qwant and more via one Docker container
  • 12-step content extraction — returns full page text with quality gates, not just links
  • Opinionated retrieval workflows — recover dead articles, capture important pages from a site, and build local docs-plus-research packs
  • Argus-owned corpus storage — runtime data goes to a writable user data directory, not your repo checkout
  • Multi-turn sessions — pass session_id for conversational context across searches
  • Score attribution — optionally show which providers contributed to each fused RRF score
  • Usage dashboard — inspect provider budgets, recent query volume, and machine-level usage at /dashboard
  • 4 search modes — discovery, research, recovery, grounding
  • Dead URL recovery — /recover-url with Wayback Machine and archive fallbacks
  • 4 integration paths — HTTP API, CLI, MCP server, Python SDK

Built for AI agent builders, RAG pipelines, and ops teams who need reliable search, capture, and local evidence without stitching APIs together.

Status: beta. The retrieval workflows and corpus model are production-oriented, but still maturing.

Status: see the public status page. Authorized maintainers can use the private argus-ops README for the latest dated reports.

Current production checkpoint — September 7, 2026: The authenticated HTTP API and MCP adapter are operational, and sampled free search works through SearXNG, Yahoo, and GitHub. DuckDuckGo is intermittent and fails closed after acquisition-policy blocks. Production readiness is ready=true, degraded; paid providers are not currently admitted until their non-secret credential-version and account-scope bindings are recorded. Browser capability and recovery metadata evidence remain open. See the current status matrix before relying on a provider.

Contents

Quickstart

Mode 1: Local CLI (zero config)

bash
pip install argus-search && argus search -q "python web frameworks"

That's it. DuckDuckGo handles the search — no accounts, no keys, no containers. You get unlimited free search from your laptop right now. Add API keys whenever you want more providers, or don't.

bash
argus extract -u "https://example.com/article"       # extract clean text from any URLargus recover-article -u "https://example.com/dead-post"argus capture-site -u "https://docs.example.com"argus build-research-pack -t "example sdk" --official-url "https://docs.example.com"

Works on any machine with Python 3.11+ — laptop, Mac Mini, Raspberry Pi, cloud VM. Nothing to host.

For MCP (Claude Code, Codex, OpenCode, Cursor, VS Code):

bash
pipx install argus-search[mcp]export ARGUS_MCP_STANDALONE=true  # explicit development-only local brokerargus mcp init --global --client all

That writes native config for Claude Code, Codex CLI, OpenCode, and Cursor. Restart the client after configuration. For manual stdio setup:

json
{"mcpServers": {"argus": {"command": "argus", "args": ["mcp", "serve"], "env": {"ARGUS_MCP_STANDALONE": "true"}}}}

Or install from the MCP Registry:

json
{  "mcpServers": {    "argus": {      "registryType": "pypi",      "identifier": "argus-search",      "runtimeHint": "uvx",      "env": {"ARGUS_MCP_STANDALONE": "true"}    }  }}

Standalone development needs no server or keys, but it must be explicitly enabled. Production MCP always delegates to an authenticated HTTP authority.

See MCP Client Setup for exact config files, verification commands, remote HTTP setup, and troubleshooting.

Mode 2: Full Stack Server

Got a Raspberry Pi running Pi-hole? A Mac Mini on your desk? An old laptop? That's enough to run the full stack — SearXNG (your own private search engine, disabled by default) plus local JS-rendering content extraction.

bash
# Optional: tell Argus it has residential egress to optimize routingexport ARGUS_EGRESS_TYPE=residentialARGUS_SEARXNG_ENABLED=true docker compose up -d    # SearXNG + Argus
What you haveWhat you get
Any machine with Python 3.11+DuckDuckGo + API providers (no server)
Home server / old laptop (4GB+)Everything — SearXNG, all providers, Crawl4AI, Obscura
Mac Mini M1+ (8GB+)Full stack with headroom
Free cloud VM (1GB)SearXNG + search providers (use residential workers for extraction)

SearXNG takes 512MB of RAM and gives you a private Google-style search engine (disabled by default — set ARGUS_SEARXNG_ENABLED=true) that nobody can rate-limit, block, or charge for. It runs alongside Pi-hole on hardware millions of people already own.

Where Argus Writes Data

Argus code and Argus runtime data are different things.

  • Code lives wherever you install or clone Argus.
  • Runtime corpus data lives in a writable user data directory resolved by platformdirs, or in ARGUS_DATA_ROOT if you override it.

Inspect the exact paths on your machine:

bash
argus paths

By default Argus writes:

  • official docs cache under the resolved docs/cache/
  • research packs under docs/research/
  • workflow run state under workflows/runs/
  • versioned workflow snapshots under snapshots/

This means Argus does not require a sibling ../docs-cache checkout. If you have an older docs-cache tree, import it once with:

bash
argus corpus import-docs-cache -s /path/to/docs-cache

Opinionated Workflows

These workflows build local artifacts, not just transient JSON responses.

Recover A Dead Article

bash
argus recover-article -u "https://example.com/old-post" -t "Example Post"

Argus searches for recovery candidates, extracts the best result, saves the recovered sources locally, and writes a citation-backed report plus manifest.

Capture The Important Parts Of A Site

bash
argus capture-site -u "https://docs.example.com"

Argus stays on-domain, uses sitemap-assisted discovery plus heuristic link scoring, saves the important pages it finds, and writes a detailed summary with references.

Build A Docs + Research Pack

bash
argus build-research-pack -t "example sdk"argus build-research-pack -t "example sdk" --official-url "https://docs.example.com"

Argus captures official docs into its local docs cache, adds non-official supporting sources from search, and writes a combined research pack with traceable artifacts.

Development

Repo development is pinned to Python 3.12. The package runtime floor is Python 3.11, the production image runs Python 3.12.3, and Python 3.13 is the compatibility CI lane. Required CI passes all three; contributors should use the uv workflow below so local verification matches the canonical lane and does not accidentally use an older system interpreter.

bash
uv sync --python 3.12 --extra dev --extra mcpuv run pytest tests/ -v --tb=short

The repo includes .python-version with 3.12 so uv, pyenv, and similar tools pick the right interpreter by default. More contributor guidance lives in CONTRIBUTING.md.

Providers

ProviderCredit typeFree capacitySetup
DuckDuckGoFree (scraped)UnlimitedNone
YahooFree (scraped)UnlimitedNone — fragile, auto-skipped if broken
SearXNGFree (self-hosted, off by default)Unlimited — 70+ engines¹Docker
GitHubFree (API)UnlimitedNone (token for higher rate limit)
WolframAlphaFree (API key)2,000 queries/monthfree key
Brave SearchMonthly recurring2,000 queries/monthdashboard
TavilyMonthly recurring1,000 queries/monthsignup
ExaMonthly recurring1,000 queries/monthsignup
LinkupMonthly recurring1,000 queries/monthsignup
Parallel AIMonthly recurring$5 credit with card on file, up to 5,000 searches/monthsignup
SerperOne-time signup2,500 creditssignup
You.comOne-time signup$20 creditplatform
ValyuOne-time signup$10 creditplatform

¹ SearXNG aggregates Google, Bing, Yahoo, Startpage, Ecosia, Qwant, Wikipedia, and 60+ more — all behind a single self-hosted endpoint. Run docker compose up -d on any machine with 512MB of free RAM.

² WolframAlpha returns computed answers (math, unit conversions, factual lookups), not web search results. It only activates in grounding and research modes. Queries it can't compute (general web searches) return empty — no error, no health penalty.

7,000+ free queries/month from recurring free-tier providers with API keys (WolframAlpha 2k + Brave 2k + Tavily 1k + Exa 1k + Linkup 1k), or up to 12,000+ when Parallel's monthly credit is available to an eligible account with a card on file. DuckDuckGo, Yahoo, and GitHub have no monthly cap. SearXNG is disabled by default (enable in .env). Routing priority: Tier 0 (free: SearXNG*, DuckDuckGo, Yahoo, GitHub, WolframAlpha) → Tier 1 (monthly recurring: Brave, Tavily, Exa, Linkup, Parallel) → Tier 3 (one-time: Serper, You.com, Valyu, SearchAPI). Budget-exhausted providers are skipped automatically.

These are package-level provider tiers and advertised quotas, not proof of current production availability. The production authority fails closed for a credentialed provider until its registration fingerprint, account scope, budget, and approved test evidence are present. A protected key value alone does not prove that the current key works; see docs/STATUS.md for the live classification.

HTTP API

All endpoints prefixed with /api. OpenAPI docs at http://localhost:8000/docs.

Local loopback calls can use the API without auth. Remote HTTP callers must send ARGUS_API_KEY as either Authorization: Bearer ... or X-API-Key: .... Privileged routes under /api/admin/* require ARGUS_ADMIN_API_KEY (or fall back to ARGUS_API_KEY if no separate admin key is configured).

bash
# Searchcurl -X POST http://localhost:8000/api/search \  -H "Content-Type: application/json" \  -d '{"query": "python web frameworks", "mode": "discovery", "max_results": 5}'
# Search with score attributioncurl -X POST http://localhost:8000/api/search \  -H "Content-Type: application/json" \  -d '{"query": "python web frameworks", "include_attribution": true}'
# Multi-turn search (conversational refinement)curl -X POST http://localhost:8000/api/search \  -H "Content-Type: application/json" \  -d '{"query": "what about async?", "session_id": "my-session"}'
# Extract content from a working URLcurl -X POST http://localhost:8000/api/extract \  -H "Content-Type: application/json" \  -d '{"url": "https://example.com/article"}'
# Recover a dead or moved URLcurl -X POST http://localhost:8000/api/recover-url \  -H "Content-Type: application/json" \  -d '{"url": "https://example.com/old-page", "title": "Example Article"}'
# Network-free process liveness (container health target)curl http://localhost:8000/api/live
# Public minimal startup and cached readinesscurl http://localhost:8000/api/startupcurl http://localhost:8000/api/ready
# Authenticated operator status, health compatibility, and budgetscurl -H "Authorization: Bearer $ARGUS_ADMIN_API_KEY" \  http://localhost:8000/api/admin/statuscurl -H "Authorization: Bearer $ARGUS_ADMIN_API_KEY" \  http://localhost:8000/api/admin/budgetscurl -H "Authorization: Bearer $ARGUS_ADMIN_API_KEY" \  http://localhost:8000/api/admin/maya-outbox/statuscurl -H "Authorization: Bearer $ARGUS_ADMIN_API_KEY" \  http://localhost:8000/api/admin/maya-outbox/dead-letters# After correcting the cause of a permanent rejection:curl -X POST -H "Authorization: Bearer $ARGUS_ADMIN_API_KEY" \  http://localhost:8000/api/admin/maya-outbox/DELIVERY_ID/recover

/api/health remains a 200 liveness compatibility route. It intentionally does not check PostgreSQL, providers, Maya, or the browser, so a dependency outage cannot cause container restart storms. See production operations for the canonical topology and operator procedures, and operational status for endpoint semantics, readiness classification, observation expiry, and safe telemetry.

Search modes
ModeUse forExample
discoveryRelated pages, canonical sources"Find the official docs for X"
researchBroad exploratory retrieval"Latest approaches to Y?"
recoveryFinding moved/dead content"This URL is 404"
groundingFact-checking with live sources"Verify this claim about Z"

Tier-based routing always applies first. Within each tier, the mode selects provider order.

Response format
json
{  "query": "python web frameworks",  "mode": "discovery",  "results": [    {      "url": "https://fastapi.tiangolo.com",      "title": "FastAPI",      "snippet": "Modern Python web framework",      "provider": "duckduckgo",      "score": 0.0164,      "score_attribution": {"duckduckgo": 0.0164},      "egress": "unknown",      "machine": null    }  ],  "total_results": 1,  "cached": false,  "traces": [    {"provider": "duckduckgo", "status": "success", "results_count": 5, "latency_ms": 312}  ]}

Each result includes url, title, snippet, domain, provider, and score. The traces array shows which providers were called and their outcomes.

When include_attribution is true, each result also includes score_attribution: a provider-to-score map that decomposes the result's Reciprocal Rank Fusion score. RRF is additive, so each provider's attribution is exactly its own rank contribution, and the values sum to score. Attribution is off by default and cached separately from non-attributed searches.

Budgets
json
{  "budgets": {    "brave": {"remaining": 1847, "monthly_usage": 153, "usage_count": 153, "exhausted": false},    "duckduckgo": {"remaining": 0, "monthly_usage": 0, "usage_count": 42, "exhausted": false}  },  "token_balances": {"jina": 9833638}}

Each provider tracks usage. Tier 1 (monthly) uses a 30-day rolling window; tier 3 (one-time) uses a lifetime counter that never resets. When a provider hits its budget, Argus skips it and moves to the next. Free providers (DuckDuckGo, GitHub) have no limit. SearXNG is free but disabled by default. Set ARGUS_*_MONTHLY_BUDGET_USD to enforce custom limits per provider.

Dashboard

Run the HTTP server and open /dashboard:

bash
argus serve# http://127.0.0.1:8000/dashboard

The dashboard shows provider budget burn, over-pace and exhausted providers, query volume for the last 30 days, usage by machine, and recent provider activity. Budget cards refresh automatically.

Set ARGUS_ADMIN_API_KEY to require dashboard login. If no admin key is set, the dashboard is open to anyone who can reach the server, which is suitable only for trusted local use.

For subpath deployment behind a reverse proxy, set ARGUS_ROOT_PATH to the external path prefix:

bash
ARGUS_ROOT_PATH=/argus argus serve

That makes dashboard redirects, links, and HTMX fragment URLs work when the proxy serves Argus at a path such as https://khamel.com/argus/.

For direct public HTTPS, the repo includes a Caddy profile:

bash
ARGUS_DOMAIN=argus.example.com [email protected] \  docker compose --profile proxy up -d

For an existing Authentik/nginx deployment, keep authentication at the proxy layer and set ARGUS_ROOT_PATH to the public prefix.

Integration

CLI

bash
argus search -q "python web framework"              # zero-config, uses DuckDuckGoargus search -q "python web framework" --mode research -n 20argus search -q "python web framework" --free        # free providers only (no paid API calls)argus search -q "python web framework" --attribution # show per-provider score attributionargus search -q "fastapi" --session my-session       # multi-turn contextargus extract -u "https://example.com/article"       # extract clean textargus extract -u "https://example.com/article" -d nytimes.com  # auth extractionargus recover-url -u "https://dead.link" -t "Title"argus doctor                                         # full setup diagnosticsargus health                                         # provider statusargus budgets                                        # budget + token balancesargus mcp check                                      # validate MCP setupargus set-balance -s jina -b 9833638                 # track token balanceargus test-provider -p brave                         # smoke-test a providerargus serve                                          # start API serverargus mcp serve                                      # start MCP serverargus mcp init                                       # add MCP config to project

All commands support --json for structured output.

How sessions work

Pass session_id to any search call. Argus stores each query and extracted URL through the same SQLAlchemy repository used by the retrieval ledger (ARGUS_DB_URL, PostgreSQL in production and SQLite for direct local use). Reusing the same session_id gives the broker context from prior queries — follow-up searches are automatically refined using earlier conversation context. Sessions persist across restarts. Omit session_id for stateless, one-shot searches.

Legacy sessions from the former budget SQLite database can be reconciled without mutating the target first:

bash
argus ledger reconcile-sessions \  --source sqlite:///argus_budgets.db \  --target "$ARGUS_DB_URL"# Review source/imported/skipped/conflicting, then repeat with --apply.

The import is idempotent: an identical existing session is skipped and a different session with the same ID is reported as conflicting.

MCP

MCP is a stateless execution adapter over the authenticated HTTP API. It does not construct providers or a broker and does not own browser, database, budget, session, health, or outbox state. Configure the adapter process with:

bash
export ARGUS_AUTHORITY_URL=http://argus-api:8000export ARGUS_AUTHORITY_TOKEN=replace-with-a-scoped-caller-token

The deployed production endpoint supports both the verified MCP 2025-11-25 compatibility contract and the MCP 2026-07-28 stateless transport revision. The newer path is one-shot and does not require an initialize handshake or Mcp-Session-Id; durable policy, budgets, sessions, and evidence remain owned by the HTTP authority. See docs/research/2026-08-11-mcp-stateless-production-authority.md for the source-backed boundary and required no-spend probes.

Option A — Local adapter (stdio)

Install the adapter on the same machine as your MCP client:

json
{  "mcpServers": {    "argus": {      "command": "argus",      "args": ["mcp", "serve"]    }  }}

Use the full path if argus isn't on PATH: "/home/you/.local/bin/argus". The adapter inherits ARGUS_AUTHORITY_URL and ARGUS_AUTHORITY_TOKEN from the client process. To run a local broker instead, development environments must explicitly set ARGUS_MCP_STANDALONE=true; production rejects it.

Works with Claude Code, Codex CLI, OpenCode, Cursor, and any stdio-based MCP client. Use argus mcp init --global --client all to write native client configs for the current machine.

Detailed client setup and verification commands live in docs/mcp-clients.md.

Option B — Remote MCP adapter (clients over Tailscale)

Run Argus on one machine, connect every client over the network. No local install on clients.

On the adapter host:

bash
export ARGUS_API_KEY=replace-with-a-long-random-secretexport ARGUS_AUTHORITY_URL=http://argus-api:8000export ARGUS_AUTHORITY_TOKEN="$ARGUS_API_KEY"argus mcp serve --transport streamable-http --host YOUR_TAILSCALE_IP --port 8001

Remote MCP credentials must also be valid scoped credentials at the HTTP authority because the adapter forwards each authenticated bearer token unchanged. For stdio, ARGUS_AUTHORITY_TOKEN is the caller credential.

To keep the HTTP API and remote MCP service running after reboot on a systemd host:

bash
cat >mcp.env <<'EOF'ARGUS_AUTHORITY_URL=http://argus-api:8000ARGUS_AUTHORITY_TOKEN=replace-with-scoped-caller-tokenARGUS_API_KEY=replace-with-the-same-scoped-caller-tokenEOFchmod 600 mcp.envARGUS_MCP_ENV_FILE="$PWD/mcp.env" scripts/install-systemd.shsystemctl status argus argus-mcp --no-pager

The installer validates the minimal adapter environment, installs it as root-only /etc/argus/mcp.env, then installs and starts both units. The MCP unit never loads the authority's .env, provider vaults, database settings, browser paths, or writable data volumes.

On each client:

ClientConfig
Claude Code{"mcpServers":{"argus":{"type":"http","url":"http://<server>:<port>/mcp","headers":{"Authorization":"Bearer <ARGUS_API_KEY>"}}}} in ~/.claude.json (global) or .mcp.json (project)
OpenCode{"mcp":{"argus":{"type":"remote","url":"http://<server>:<port>/mcp","enabled":true,"headers":{"Authorization":"Bearer <ARGUS_API_KEY>"}}}} in ~/.config/opencode/config.json (global) or .opencode/opencode.json (project)
CursorSame as Claude Code — reads .mcp.json
Codex CLI[mcp_servers.argus] section in ~/.codex/config.toml with url and bearer_token_env_var = "ARGUS_API_KEY" — export that variable in the shell that launches Codex; argus mcp init never writes the token to disk
Gemini CLIgemini mcp add argus http://<server>:<port>/mcp -t http -H "Authorization: Bearer <ARGUS_API_KEY>"
Antigravity{"mcpServers":{"argus":{"serverUrl":"http://<server>:<port>/mcp","headers":{"Authorization":"Bearer <ARGUS_API_KEY>"}}}}

With Tailscale, <server> is your machine's Tailscale IP (e.g. 100.x.x.x). One server, every machine on your mesh gets search.

One-command provisioning:

bash
# Load secrets, then push config to any machine:eval $(secrets decrypt argus | grep -E 'ARGUS_REMOTE_URL|ARGUS_API_KEY' | sed 's/^/export /')
curl -s https://raw.githubusercontent.com/Khamel83/argus/main/scripts/provision-mcp-client.sh | bash -s local              # this machine; uses local stdio if argus is installedcurl -s https://raw.githubusercontent.com/Khamel83/argus/main/scripts/provision-mcp-client.sh | bash -s [email protected]    # remote machine

The script writes Claude/Cursor, Codex, and OpenCode configs on the target. Local stdio does not require an MCP listener key, but the adapter still requires its scoped ARGUS_AUTHORITY_TOKEN. Remote MCP mode requires ARGUS_REMOTE_URL and ARGUS_API_KEY. Requires Python 3.

argus mcp init also generates configs automatically:

bash
argus mcp init --global              # local stdio adapter for Claude Code + OpenCode + Cursorargus mcp init --client codex        # local stdio for Codex (writes ~/.codex/config.toml)argus mcp init --client opencode     # local stdio for OpenCodeARGUS_REMOTE_URL=http://argus.local:8271 ARGUS_API_KEY=... argus mcp init --global --client allargus mcp init --client gemini       # prints gemini mcp add commandargus mcp init --global --client all # everything above

Release Status

Pushing main does not publish PyPI. Package and MCP Registry publication happens through the GitHub publish workflow on release creation or manual dispatch. See docs/releasing.md for version sync, preflight checks, and publish verification.

Transports: stdio (default local adapter), sse (legacy remote), and streamable-http (modern remote, "type":"http" in config). Remote MCP transports require ARGUS_API_KEY; every transport delegates execution to ARGUS_AUTHORITY_URL.

Available tools:

  • HTTP-backed stdio and remote MCP: search_web, extract_content, recover_url, expand_links, search_health, search_budgets, recover_dead_article, capture_site, and build_research_pack
  • Explicit standalone development additionally exposes local-only test_provider, cookie_health, valyu_answer, path/file tools, and resources. These are intentionally absent from production adapters.

search_web accepts free_only=true to restrict results to free (tier-0) providers only, and include_attribution=true to include per-provider score attribution in the Markdown response.

Using Argus from MCP vs HTTP

Two transports, one rule: agents use MCP, everything else uses HTTP.

  • MCP (argus mcp serve) — a stateless authenticated adapter for AI harnesses that speak MCP natively. Core tools delegate to the HTTP authority: search_web, extract_content, recover_url, expand_links, and workflow starts. MCP restarts cannot fork accounting, sessions, health, or outbox state.
  • HTTP (POST /api/search, POST /api/extract, POST /api/workflows/...) — for scripts, cron jobs, and service integrations (including Maya). Send a scoped caller credential in Authorization; body "caller" values are diagnostic labels and cannot override the authenticated identity.

The cross-service transport and role contract for the wider fleet (Maya / Hermes / Argus) is canonical in Maya's architecture documentation.

Python

Direct broker and extraction imports are a standalone development convenience. Production Python callers use the authenticated HTTP API so all execution and durable accounting remain in one authority.

python
from argus.broker.router import create_brokerfrom argus.models import SearchQuery, SearchModefrom argus.extraction import extract_url
broker = create_broker()
response = await broker.search(    SearchQuery(query="python web frameworks", mode=SearchMode.DISCOVERY, max_results=10),    compute_attribution=True,)for r in response.results:    print(f"{r.title}: {r.url} (score: {r.score:.3f})")    print(r.score_attribution)
content = await extract_url(response.results[0].url)print(content.title)print(content.text)

Content Extraction

Argus tries up to twelve methods to extract content from any URL: auth extraction for paywalls, then local extractors (trafilatura, Crawl4AI, Obscura, Playwright, residential IP), then external APIs (Jina, Valyu Contents, Firecrawl, You.com, Wayback, archive.is). Each attempt is quality-checked for completeness and garbage output. See docs/providers.md for the full extractor comparison.

Completeness assessment runs automatically after every successful extraction. Argus scores five signals — trailing ellipsis, feed truncation markers ("Read more", WordPress RSS footers), mid-sentence endings, abrupt final paragraphs, and suspicious round word counts — and returns is_complete, completeness_confidence, and truncation_type alongside the text. When confidence is ≥ 85%, Argus continues trying the next extractor rather than returning a partial result; this means a trafilatura fetch that ends with "..." will automatically fall through to Playwright, Jina, Wayback, etc. Callers that already have text (e.g. RSS feed items) can use POST /api/assess-content to check completeness without triggering extraction.

Obscura (optional) is a lightweight Rust headless browser (~70MB binary, 30MB RAM) with built-in stealth mode — it sets navigator.webdriver=undefined, randomizes canvas/GPU/audio fingerprints per session, and blocks 3,520 tracker domains. This directly addresses bot detection on JS-heavy and anti-scraping sites that block standard Playwright/Chrome. No API key, no rate limit — fully local.

Two ways to use it:

ModeSetupWhat you get
CLI extraction stepInstall binary on $PATHArgus auto-detects it; stealth browser as fallback step before Playwright
CDP backend for PlaywrightRun obscura serve --stealth --port 9222, set ARGUS_OBSCURA_CDP_URL=ws://127.0.0.1:9222Playwright uses Obscura as its browser engine — stealth + 30MB vs 200MB + DOM-to-Markdown output

Install the binary: github.com/h4ckf0r0day/obscura/releases

Extract gets the full text of a working URL and tells you whether that text is complete. Recover-URL finds alternatives when a URL is dead, paywalled, or radically changed.

Architecture

Caller (CLI/HTTP/MCP/Python) → SearchBroker → tier-sorted providers → RRF ranking → response                                     ↕ SessionStore (optional)                            Extractor (on demand) → 12-step fallback chain with quality gates
ModuleResponsibility
argus/broker/Tier-based routing, ranking, dedup, caching, health, budgets
argus/providers/Provider adapters (one per search API)
argus/extraction/12-step URL extraction fallback chain with quality gates
argus/sessions/Multi-turn session store and query refinement
argus/api/Authenticated production execution authority
argus/cli/HTTP caller in production; direct execution in development
argus/mcp/Stateless MCP-to-HTTP adapter
argus/persistence/Shared PostgreSQL authority state; SQLite standalone development
argus/operations/Cached readiness, typed dependency observations, process identity, bounded metrics

Add new providers or extractors with a single adapter file. See CONTRIBUTING.md for the interface.

How a Query Works

query arrives → cache? → build provider queue → execute sequentially → RRF fuse → dedup → respond
  1. Cache check. SearchCache hashes the normalized query, mode, and whether attribution was requested (SHA256). Hit returns immediately with a TTL of 168 hours (7 days).

  2. Provider queue. resolve_routing() takes the mode-specific preference list and stable-sorts by tier: tier 0 (free) first, tier 1 (monthly) next, tier 3 (one-time) last. Example for discovery mode:

    searxng → duckduckgo → yahoo → github → brave → exa → tavily → linkup → parallel → serper → you → valyu
  3. Sequential execution with gates. Each provider is checked in order. Four gates must pass before an API call:

    • Config — is the provider enabled and configured (API key present)?
    • Health — has it failed 5+ consecutive times (triggers 60-minute cooldown)?
    • Budget — for tier 1+: is the budget exhausted? For tier 1 (monthly), pacing checks if the 7-day usage rate would drain the remaining budget in under a week — empty days bank headroom. For tier 3 (one-time), a lifetime counter gates access — exhaustion is the sole check.
    • Execute — the actual HTTP call. Successes reset failure counters; failures increment them.
  4. RRF fusion. Results from all queried providers are merged using Reciprocal Rank Fusion (k=60). Each result's score is the sum of 1/(k + rank) across every provider that returned it. Results appearing in multiple providers rank higher.

  5. Dedup and truncate. URLs are normalized (stripped www., tracking params like utm_*, trailing slashes) and deduplicated. The merged list is truncated to max_results (default 10).

  6. Cache and persist. The authority writes the final response to its in-memory cache and configured SQL repository (PostgreSQL in production, SQLite for standalone development). Search results and extractions include provenance metadata (egress, machine, source_type) for downstream audit. Existing databases are upgraded additively at startup.

Configuration

All config via environment variables. See .env.example for the full list. Limited API-key providers are opt-in: set both the API key and ARGUS_<PROVIDER>_ENABLED=true. Missing keys degrade gracefully — providers are skipped, not errors.

When running from the repo, Argus now auto-loads .env and .env.local (without overriding already-exported environment variables). Disable this behavior with ARGUS_AUTOLOAD_DOTENV=false.

VariableDefaultDescription
ARGUS_WORKER_BIND127.0.0.1:8273Legacy development worker bind; requires explicit ARGUS_ENV=development
ARGUS_EGRESS_SHARED_SECRET(empty)Required bearer secret for every legacy worker execution request, including loopback
ARGUS_NODE_ROLEprimaryProduction authority is primary; adapters are caller; direct/worker execution is development-only
ARGUS_AUTHORITY_URL—HTTP API base URL required by production CLI and MCP adapters
ARGUS_AUTHORITY_TOKEN—Scoped caller token for production CLI and MCP adapters
ARGUS_MCP_STANDALONEfalseExplicit development-only local MCP execution
ARGUS_EGRESS_TYPEunknownresidential, datacenter, or unknown
ARGUS_RESIDENTIAL_POLICYfallbackoff, fallback, prefer_on_datacenter, prefer_for_domains, or always
ARGUS_SEARXNG_ENABLEDfalseSet true when you have a SearXNG Docker container
ARGUS_SEARXNG_BASE_URLhttp://127.0.0.1:8080SearXNG endpoint
ARGUS_SEARXNG_RESIDENTIAL_BASE_URL—Remote residential SearXNG endpoint (e.g. over Tailscale)
ARGUS_<PROVIDER>_ENABLEDfalse for limited API-key providersOpt in to providers that consume limited credits or quotas
ARGUS_BRAVE_API_KEY—Brave Search API key
ARGUS_SERPER_API_KEY—Serper API key
ARGUS_TAVILY_API_KEY—Tavily API key
ARGUS_EXA_API_KEY—Exa API key
ARGUS_LINKUP_API_KEY—Linkup API key
ARGUS_PARALLEL_API_KEY—Parallel AI API key
ARGUS_YOU_API_KEY—You.com API key
ARGUS_VALYU_API_KEY—Valyu API key (search, contents, answer)
ARGUS_FIRECRAWL_API_KEY—Firecrawl API key (content extraction)
ARGUS_GITHUB_API_KEY—GitHub token (higher rate limit)
ARGUS_DATA_ROOTplatformdirs user data dirOverride the Argus runtime corpus root
ARGUS_*_MONTHLY_BUDGET_USDprovider-specificQuery-count budget for most providers; USD budget for Valyu
ARGUS_CRAWL4AI_ENABLEDfalseEnable Crawl4AI extraction step
ARGUS_GATEWAY_URL—OpenAI-compatible gateway (for example g2k, http://<host>:4312/v1) used by POST /api/workflows/search-and-summarize to write the answer. Unset: the deterministic extractive summary is used
ARGUS_GATEWAY_KEY—Bearer token for that gateway. Use a workload token provisioned for Argus, never a shared endpoint token
ARGUS_GATEWAY_MODELautoModel sent to the gateway, for example auto, gemini-web/gemini-flash or chatgpt-web/gpt-5-6-thinking-medium
ARGUS_GATEWAY_MODEgroundedgrounded answers only from captured sources. expand also lets the model research on its own and returns anything beyond the sources as an unverified "Additional leads" section. If the model does not separate them, the whole answer is marked unverified and carries no citations
ARGUS_GATEWAY_TIMEOUT120Seconds to wait for the gateway before falling back to the extractive summary
ARGUS_MIN_GOOD_RESULTS3Distinct domains required before one-time-credit providers are skipped. Free and monthly providers always run
ARGUS_GUARD_ENABLEDtrueRunaway-caller guard on POST /api/search: repeat/burst limits run free-only, the daily cap returns 429
ARGUS_GUARD_REPEAT_LIMIT / _BURST_LIMIT / _DAILY_LIMIT3 / 60 / 300Same query per 10 min, calls per 10 min, calls per 24 h
ARGUS_GUARD_MODEL_NAME—Model identifier sent with guard review requests (required by most OpenAI-compatible servers)
ARGUS_GUARD_MODEL_URL—Optional loopback/private OpenAI-compatible endpoint that reviews tripped callers; can only double the burst limit for an hour
ARGUS_YOU_CONTENTS_ENABLEDfalseEnable You.com Contents API extraction
ARGUS_OBSCURA_CDP_URL—Obscura CDP endpoint (e.g. ws://127.0.0.1:9222) — makes Playwright use Obscura as its browser engine
ARGUS_OBSCURA_TIMEOUT_SECONDS20Timeout for Obscura CLI subprocess calls
ARGUS_AUTH_BROWSER_DOMAINS—Temporary, off by default (#148). Comma-separated paywall domains whose cookie-backed authenticated_content HTTPS browser requests may run without a browser-network attestation. Applies only when no attestation is present; same-origin guards still apply
ARGUS_CACHE_TTL_HOURS168Result cache TTL
ARGUS_BIND_HOST127.0.0.1Host used by argus serve unless --host is passed
ARGUS_PORT8000Port used by argus serve unless --port is passed
ARGUS_AUTOLOAD_DOTENVtrueAuto-load .env / .env.local from cwd and repo root for CLI/API/MCP processes
ARGUS_API_KEY—Required for non-local HTTP API and remote MCP callers
ARGUS_ADMIN_API_KEY—Enables dashboard login and admin API authentication
ARGUS_ACCEPTED_OPERATION_AUTHORITYlegacyAtomic authority selection. evidence activates the registered planner, readiness, evidence repository, extraction finalizer, and HTTP presenters as one unit
ARGUS_ALLOWED_HOSTS—Exact comma-separated HTTP Host allowlist; required for a remote production listener
ARGUS_ALLOWED_ORIGINS—Exact comma-separated browser Origin allowlist. Set explicitly, including an empty value, for remote production
ARGUS_RETRIEVAL_SESSION_SECRET—Stable random secret of at least 32 characters used to bind v2 retrieval sessions to authenticated principals
ARGUS_ORGANIZATION_POLICY_VERSION1Stable organization-policy identity included in accepted execution cohorts
ARGUS_ROOT_PATH—Public subpath prefix for dashboard links and redirects, e.g. /argus
ARGUS_MAYA_CAPTURE_URL—Maya's dedicated Argus retrieval-capture endpoint; delivery stays disabled when unset
ARGUS_MAYA_CAPTURE_TOKEN—Dedicated shared secret for Maya capture delivery; never reuse the generic Maya ingest token
ARGUS_MAYA_OUTBOX_BATCH_SIZE20Maximum durable captures claimed by one delivery pass (bounded to 100)
ARGUS_MAYA_ACKNOWLEDGED_RETENTION_DAYS7Days to retain acknowledged capture bodies before preserving audit metadata only

/api/v2/* is additive and returns a canonical version-2 envelope. It remains fail-closed with unready while the evidence authority is disabled. Unsafe Host, Origin, credential, media-type, and body-size combinations are rejected before provider, extractor, session, or persistence work. Version-1 routes retain their established response shapes.

When Not To Use Argus

Argus is best when you need search, capture, provenance, and local artifacts together.

Avoid it when:

  • you only need one search API and do not need fallback or budget controls
  • you only need a one-off page scrape with no persistent corpus or report output
  • you need an end-user search UI rather than backend retrieval infrastructure
  • you need fully deterministic summarization with no heuristic or LLM-assisted steps

FAQ

How is this different from calling Tavily/Serper directly? Argus calls them for you — plus 13 other providers. You get one ranked, deduplicated result set instead of managing multiple API keys and stitching results together. Free providers are tried first, so you only burn credits when needed.

Can I run only one provider? Yes. Set only the API key for the provider you want. All others are silently skipped. For zero-config, just install and go — DuckDuckGo + Yahoo handle search with no keys.

Do I need Docker? No. pip install argus-search works immediately on any machine with Python 3.11+. Docker is only needed for SearXNG (set ARGUS_SEARXNG_ENABLED=true in .env) or Crawl4AI (local JS rendering).

Which Python version should contributors use? Use Python 3.12 for repo development and verification: uv sync --python 3.12 --extra dev --extra mcp then uv run pytest tests/ -v --tb=short. The published package still supports Python 3.11+.

What is the safest way to deploy Argus on a network? Use Tailscale or another private network, bind explicitly to the trusted interface, set ARGUS_API_KEY, and reserve /api/admin/* for ARGUS_ADMIN_API_KEY. Treat direct internet exposure as an advanced mode behind a reverse proxy.

License

MIT — see CHANGELOG.md for release history.

來源:README.md,提交 0e3821e

工具

0
SourceWeft 曾在隔離沙箱中以無憑證方式啟動該軟體包一次,以列出其工具,但未能完成:該軟體包類型無法在沙箱中執行。

版本歷史

2
  1. v1.6.5最新Oct 22, 2026
  2. v1.6.4Sep 16, 2026

評分

已滿足適用檢查項的 50%

該評分衡量此收錄條目的完整程度,依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價,也不表示伺服器安全或目前可用:可連通性是帶日期的檢查結果,評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通25 分

    請保持端點能夠回應 MCP 握手;SourceWeft 每 30 天探測一次。

  • 已宣告認證方式10 分

    請在註冊中心條目中宣告憑證請求標頭,或保持端點能夠回應探測,以便判定其認證方式。

  • 已列出工具15 分

    請讓伺服器在無憑證時回應 tools/list,或發布可在沙箱中無憑證啟動的軟體包。

  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分

其他 Argus 整合