Tinysearch

TinySearch

[TinySearch]

Spend tokens on answers, not webpages.

TinySearch searches, crawls, and reranks the web locally, then gives your agent only the evidence worth putting in its context.

Documentation · Quick start · Python · Discord

[Website] [PyPI version] [PyPI Downloads] [Tests] [License: MIT] [Python] [Release] [Last Commit] [Docker Pulls] [Discord] [MCP Server] [FastAPI]

TinySearch is a self-hosted web-research tool for AI agents. It searches the web, reads the best pages, removes low-value content, and returns compact evidence with source URLs.

Your model receives the useful passages instead of paying to process entire webpages.

TinySearch is part of TinySuite, a suite of focused tools designed to make agentic operations cheaper by minimizing token usage through smart retrieval, selection, and context-management techniques.

[TinySearch returning source-grounded web evidence to an AI agent]

Choose a tier

Tiers 1 and 2 need no search service. Tier 3 adds a dedicated SearXNG service, persistent model storage, and a network MCP endpoint. See the installation guide for the Docker setup.

The expensive part of agent research is context

A search result is not yet useful evidence. Agents often have to open several pages, ingest navigation and boilerplate, and spend paid input tokens deciding which passages matter.

TinySearch moves that work in front of the model:

flowchart LR    A[Question] --> B[Search and crawl]    B --> C[Local hybrid reranking]    C --> D[Compact evidence<br/>with source URLs]    D --> E[Your agent]

That lowers cost in three ways:

• Smaller model context. Only the best-ranked evidence chunks are returned, within a controlled evidence budget.
• No metered search API required by default. TinySearch can search through DDGS without a paid search provider.
• Local retrieval by default. ONNX embeddings and hybrid reranking run on your machine instead of creating embedding API charges.

Search broadly. Read locally. Pay the model only for the evidence that matters.

This is retrieval, not summarization: TinySearch selects the passages worth keeping with local BM25 and embedding rerank, it doesn't run a model over the page to rewrite or condense it. Every returned chunk is the original page text, unedited, so what you cite is what the page actually said. That keeps the pipeline fast and free to run locally, at the cost of not compacting as aggressively as a dedicated reduction model could. A learned reduction step is a direction we may explore later; it isn't part of TinySearch today.

Actual savings depend on the pages, evidence limits, client model, and provider pricing. TinySearch reduces the web content sent to the model; it does not control what the client does with that evidence afterward.

[Benchmark: TinySearch uses 64% fewer tokens than a naive search-and-fetch agent across 8 research queries, cutting modeled input cost per 1,000 queries from $55.08 to $20.03 at $3 per million tokens]

The cost panel uses an illustrative $3.00 per million input-token rate and excludes search, crawling, model output, and downstream agent use.

The naive baseline isn't a strawman product, it's the same pages TinySearch crawled for each query, fed to the model unfiltered, the way a generic "search, then fetch the page" tool (a plain web-search-plus-fetch loop, the kind built into most coding agents) would. Measured against the current recommended flow (search then scrape_urls) and counted on the actual MCP tool-result text, TinySearch's primary interface. Reproduce or rerun it yourself:

python scripts/benchmark_token_savings.py --json-out report.json

Quick start

With uv installed, add TinySearch to any MCP client:

{  "mcpServers": {    "tinysearch": {      "command": "uvx",      "args": [        "--python",        "3.12",        "--from",        "tinysuite-search[server]",        "tinysearch"      ]    }  }}

The client launches TinySearch over stdio when it needs it. No repository clone, hosted account, or paid search key is required.

Fast search starts without Chromium or an embedding model. The first scrape initializes Chromium; focused scraping also initializes the configured embedding model. Pre-warm both ahead of time if you will use those workflows:

uvx --from "tinysuite-search[server]" tinysearch setup

The MCP and FastAPI servers keep the scraper browser warm between nearby requests, then close it after browser_idle_shutdown_seconds. Direct Python calls retain their short-lived, caller-owned lifecycle.

The interactive browser_* tools give every MCP client connection its own browser and page, so clients sharing one server never see each other's tabs or cookies. At most browser_max_sessions (default 4) are open at once; a new caller waits up to browser_session_wait_seconds (default 30) for one to close, explicitly or after its idle timeout, then gets an error. The FastAPI /browser/* endpoints share one browser, as before.

[TinySearch CLI setup and first run in a terminal]

Prefer Docker, a remote MCP endpoint, or a source checkout? Follow the installation guide.

The MCP tools

TinySearch deliberately stays focused. It is a retrieval layer, not another agent, chat interface, hosted search product, or permanent web index.

See the complete MCP tool reference for parameters and response contracts.

What your agent gets

TinySearch does not spend another model call writing the final answer. The recommended flow is search for lightweight discovery, then scrape_urls for the pages worth reading.

Search returns structured JSON. Use one item for a simple lookup; add multiple items only for independent subquestions or source strategies. domains is a hard positive source restriction and accepts a domain plus its subdomains:

{"items":[{"query":"Form 8-K Tesla","domains":["sec.gov"]}]}

Each search item reports its own results and compact backend attempts. A zero result response is distinct from a blocked, unavailable, or invalid backend. scrape_urls returns selected Markdown evidence and separate related-link navigation candidates, each with independent configured token ceilings.

MCP still uses its standard JSON-RPC transport envelope, including protocol-level errors and optional structuredContent. Python and FastAPI keep their structured JSON contracts for applications that need to store, inspect, or transform the evidence.

How it works

1. search returns backend-ordered titles, URLs, previews, upstream dates, and backend outcomes without starting Chromium or an embedding model.
2. scrape_urls reads one to five known pages concurrently. Omit an item's query or use "*" to keep clean Markdown in page order within the configured token budget.
3. Supply a focused item query when TinySearch should chunk and hybrid-rank that page before returning evidence.
4. The browser_* tools step in only when scrape_urls can't reach the content because it needs interaction. See Browser automation.

Python library

TinySearch also works as a regular Python package:

pip install tinysuite-search

Optional OpenTelemetry export

TinySearch emits vendor-neutral traces and metrics only when OpenTelemetry is explicitly configured. Normal library, MCP, and FastAPI behavior is unchanged when telemetry is not installed or not configured.

Install the optional exporter support for a standalone MCP server:

uvx --from "tinysuite-search[server,telemetry]" tinysearch

The official Docker image includes the same optional support. Set standard OTel variables on either deployment; the common endpoint enables traces and metrics:

OTEL_SERVICE_NAME=tinysearch \OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 \OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \tinysearch serve

http/protobuf is the default and grpc is also supported. Use OTEL_TRACES_EXPORTER=none, OTEL_METRICS_EXPORTER=none, or OTEL_SDK_DISABLED=true to disable telemetry. Standard resource, header, timeout, sampler, and signal-specific endpoint settings are passed through to the OpenTelemetry SDK; treat OTEL_EXPORTER_OTLP_HEADERS as a secret.

TinySearch exports operation/stage timing, outcomes, counts, backend state, browser use, token counts, and embedding model metadata. It never exports queries, URLs, domains, prompts, documents, snippets, request headers, credentials, configuration paths, raw errors, exception stacks, MCP arguments, or MCP results. Direct Python-library users configure their own OTel provider; TinySearch only auto-configures its standalone MCP and FastAPI server entry points.

import asynciofrom tinysearch import scrape_urls, search

async def main():    results = await search([{"query": "Python async tasks"}])    print(results["items"][0]["results"])
    page_url = results["items"][0]["results"][0]["url"]    evidence = await scrape_urls([{        "url": page_url,        "query": "How does asyncio cancellation work?",    }])    print(evidence["results"])

asyncio.run(main())

The Python API returns stable, JSON-serializable results. search accepts one to five items and uses the configured per-item result limit. scrape_urls accepts a per-call max_tokens budget (4,000 by default); omit an item's scrape query or use "*" for page-order mode. Rendering structured evidence into an LLM prompt is explicit, so applications can store, inspect, transform, or budget the result first.

The optional FastAPI app mirrors these surfaces. POST /search accepts the same batch JSON contract. POST /scrape accepts one to five { "url", "query" } items and always returns structured per-item outcomes. POST /browser/navigate and POST /browser/act mirror the two MCP browser tools and return their accessibility view as { "result": "..." }. The app also exposes /health, /current_datetime, and read-only /config; configuration writes require explicit environment opt-in.

Activity and management views

TinySearch records nothing about what it searched unless you ask it to. TINYSEARCH_ACTIVITY turns recording on:

Records older than TINYSEARCH_ACTIVITY_RETENTION_DAYS (default 7) are deleted, and only the newest TINYSEARCH_ACTIVITY_MAX_EVENTS (default 5000) are kept. Callers are stored as salted hashes. Everything lives in one SQLite file, TINYSEARCH_ACTIVITY_DB (default: activity.db in the user data directory). On macOS and Linux it, and saved browser cookies, are created owner-only (0600), in a folder created 0700; on Windows the per-user AppData folder already keeps them private.

TINYSEARCH_MANAGEMENT=on exposes these records to a client UI, such as TinyWebUI's MCP panel, as MCP resources following the tinysuite.manage/1 contract (manifest tinysearch://manage/manifest, media type application/vnd.tinysuite.manage+json):

• Overview, Domains, Blocking rules (instance-wide): health and counts, discovered versus fetched domains, configured blocked_domains, and the built-in SSRF rules. A search item's own domains narrow that one search; they are not a blocklist.
• My searches, My pages (per caller): that caller's own history in history or content mode. Each entry in My searches opens to its query, URLs and outcome, and in content mode to what the model was shown. A clear_search_history tool, added only in those modes, deletes it all.

Who may read what:

• A local stdio client is the operator and sees everything.
• Over HTTP, the instance views need Authorization: Bearer <TINYSEARCH_MANAGEMENT_TOKEN> (24+ characters).
• Over HTTP, a caller's own views need a trusted identity: the TINYSEARCH_TRUSTED_USER_HEADER header, believed only from peers in TINYSEARCH_TRUSTED_PROXY_CIDRS that also send X-TinySearch-Proxy-Token: <TINYSEARCH_TRUSTED_PROXY_TOKEN> (at least 32 characters, required with the header), checked on every message. The address alone does not prove the proxy sent a request: another container on that network, or a local program when TinySearch listens on loopback, could otherwise name any user. A request without the right token is treated as a caller with no identity: its activity stays aggregate.

Management over HTTP with neither a token nor a trusted identity refuses to start.

Search backends

TinySearch selects a web-search backend from config, so you can start with no search service and add one later without changing code.

• "ddgs" (native default): queries the ddgs package's automatic backend selection in-process. No SearXNG deployment required.
• "searxng" (Docker default): queries a self-hosted SearXNG instance. Falls back to ddgs on backend failure unless search_backend_fallback is set to false.
• "duckduckgo": skips SearXNG and queries ddgs in DuckDuckGo-only mode.
• "auto": tries SearXNG, then falls back to ddgs on any backend failure.

Set the BRAVE_SEARCH_API_KEY environment variable to add Brave's official Web Search API as a keyed fallback for the ddgs and duckduckgo backends. Brave is only consulted when the primary call errors or returns no results.

Full key reference, SearXNG JSON-output setup, and Compose details live in the configuration reference.

Browser automation

scrape_urls is a static fetch. It cannot see content that JavaScript renders after load, content behind a cookie interstitial, a "load more" control, or a client-side search UI. For those pages TinySearch drives a real browser.

This costs nothing extra to install. TinySearch already scrapes pages with Playwright and installs its Chromium for that, so the browser tools reuse the same driver and the same browser: no second runtime, no second browser, no child process.

Two tools: browser_navigate and browser_act, the second folding look, click, type, wait_for, tabs, and close behind one action parameter.

That split is deliberate. MCP has no way to group or nest tools -- tools/list is flat and every schema is re-sent to the model on every request -- so seven separate browser tools would dominate the server's schema. Publishing the entry point as its own tool and folding one page session's lifecycle behind a dispatcher keeps the whole server's schema small across five tools.

There is no separate find tool, because finding is not a sibling of clicking -- it is a filter on the result. Both tools take one find argument, tried as a regex first (so "a|b" works directly) and falling back to a literal, case-insensitive substring match for text that isn't valid regex -- narrowing the return value to the matching nodes and their context instead of the whole tree. That is what lets one call both act and report: a click that reveals a table comes back as the table, so the agent never spends a second call narrowing the first one's answer. An earlier version split this into find and find_regex; that cost a wasted round trip whenever a model reached for alternation syntax on the plain-substring parameter and got "no matches" instead of a hint, so the two were merged.

The model reads a compact accessibility tree where each node carries a stable ref, names one, and TinySearch acts on it with genuine browser input events:

browser_navigate  -> url: "...", find: "Accept"   -> "- button \"Accept all\" [ref=e79]"browser_act       -> action: "click", target: "e79", find: "Results"

Nothing synthesizes DOM events or invents CSS selectors, and click/type accept only a ref the model actually observed, never a raw selector.

Three deliberate choices:

• No tool can execute code. There is no evaluate tool, and none that fills forms, uploads, or drags. A page that injects instructions into its own rendered text has nothing dangerous to reach for, because the capability is absent rather than discouraged.
• find is the token lever, depth the fallback. Any call that returns a view takes find, cutting it to the matching nodes and their context. When no filter can name the target, depth returns a shallower but still valid tree rather than a truncated string -- on a large page ~700 characters versus ~33,000.
• Cookies persist, sessions don't. Set browser_storage_state_path and a consent banner accepted once is not paid for on every later navigation. Sessions stay isolated, so no browser profile lock is taken and concurrent clients do not conflict. That file is a server-side path, never exposed to a model or over HTTP. On a server that tells callers apart (a trusted identity is configured), each caller keeps their own file next to it, named by a salted hash, so one person's logins never reach another's browser. A caller without an identity, and the page fetcher's browser that serves everyone, use none.

To turn the tools off entirely, set "browser_backend": "off"; they are then removed from the tool list rather than merely refusing to run.

External browser over CDP

To drive a browser you operate separately, set its Chrome DevTools Protocol endpoint. It is used by both the scrape pipeline and the browser tools:

{  "browser_cdp_url": "http://browser:9222"}

Server processes also accept TINYSEARCH_BROWSER_CDP_URL. The external browser owns its executable, profile, proxy, and fingerprint configuration; TinySearch does not select or install a particular browser backend.

Treat a CDP endpoint as privileged remote control of the browser. Keep it on a private network or loopback interface, require authentication when it crosses a host boundary, and do not expose port 9222 directly to the public internet. When TinySearch itself runs in Docker, localhost refers to the TinySearch container, so use an endpoint reachable from that container.

browser_backend, browser_cdp_url, and browser_storage_state_path are operator-managed and cannot be changed through the HTTP PUT /config endpoint, even when configuration writes are enabled. Set them in the startup environment (TINYSEARCH_BROWSER_BACKEND and friends) or the file selected by TINYSEARCH_CONFIG_PATH, then restart TinySearch. HTTP clients can continue updating other settings by omitting these fields from their partial update.

Proxies

To route crawling through a proxy, set browser_proxy_server:

{  "browser_proxy_server": "http://gateway.proxyprovider.com:8000",  "browser_proxy_username": "your-username",  "browser_proxy_password": "your-password"}

http, https, and socks5 proxy URLs are supported, matching what Playwright accepts. If your provider hands out several fixed gateway endpoints, comma-separate them and TinySearch round-robins one per new browser context:

{  "browser_proxy_server": "http://gw1.proxyprovider.com:8000, http://gw2.proxyprovider.com:8000"}

Use browser_proxy_bypass for a comma-separated list of hosts that should skip the proxy. These four fields are operator-managed like the CDP settings above -- settable via TINYSEARCH_BROWSER_PROXY_SERVER, TINYSEARCH_BROWSER_PROXY_USERNAME, TINYSEARCH_BROWSER_PROXY_PASSWORD, and TINYSEARCH_BROWSER_PROXY_BYPASS, or the config file, but never over HTTP.

Why TinySearch

• No vendor in the loop. No TinySearch account, no required API key, no per-request billing, no analytics service or hosted scraped-data cache. The infrastructure you'd otherwise pay a search API for runs on your machine.
• Source-grounded by construction. Every evidence chunk is the original page text, still attached to its originating URL, so a claim in your agent's answer traces back to one specific passage instead of stopping at "the vendor's model said this."
• Built around token efficiency. Page selection and passage selection happen locally, before content enters model context.
• Useful without paid infrastructure. DDGS search and local ONNX embeddings are the defaults.
• Bring your own stack when needed. SearXNG and OpenAI-compatible embedding providers remain optional.
• Works where agents already work. Use MCP over stdio, Streamable HTTP, Python, FastAPI, or Docker.

Part of TinySuite

TinySuite is a product suite built around one idea: agents should spend tokens on useful work, not operational overhead.

Each tool focuses on a different part of the agent workflow and uses targeted techniques to reduce unnecessary context before it reaches the model. TinySearch handles the web-research layer by turning pages into a small, ranked, source-grounded evidence packet.

Documentation

The README is the product overview. Detailed setup and operational material lives in the TinySuite documentation:

• TinySearch overview and installation
• Configuration reference
• MCP tools
• Troubleshooting

The repository also contains an annotated example configuration at configs/tinysearch_config.json.

When not to use TinySearch

TinySearch is intentionally lightweight. Use a commercial search API, persistent crawler, or full search index when you need:

• guaranteed search coverage or an SLA
• large-scale or scheduled indexing
• long-term page storage and change history
• enterprise observability and access controls

Development

git clone https://github.com/TinySuiteHQ/TinySearchcd TinySearchpython -m venv .venvsource .venv/bin/activatepip install -e ".[server]"python -m unittest discover tests

TinySearch supports Python 3.12 and newer. CI tests Python 3.12, 3.13, and 3.14 across Linux, macOS, and Windows.

Entrypoints

• tinysearch.search and tinysearch.scrape_urls: structured Python API
• tinysearch.get_current_datetime: structured UTC date and time
• tinysearch.to_prompt: pure structured-evidence prompt renderer
• tinysearch mcp: stdio MCP server (also the no-argument default)
• tinysearch serve: Streamable HTTP MCP server
• tinysearch.servers.fastapi_server:app: optional FastAPI application

Community

Questions, ideas, and bug reports are welcome:

• Join the TinySearch Discord
• Open a GitHub issue
• Email the maintainer

Privacy and license

TinySearch reads public pages and returns selected excerpts to the calling client. Search, crawling, local embeddings, and reranking can run without sending page content to an embedding provider. If you choose an OpenAI-compatible embedding backend, that provider receives the text sent for vectorization.

TinySearch is available under the MIT License. Downloaded model weights remain subject to their respective model-card licenses. See NOTICE for third-party distribution details.