DeepSift Search

io.github.pavanmulkaPublished by GitHub account pavanmulkav0.1.0Updated Nov 5, 2026

Live Google web search for AI agents that returns only the passages that answer the question.

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingKnowledge & Memory

Overview

AI-generated overview

Hosted web search MCP server that runs Google searches and returns only the page passages that answer the question, with read, map, crawl and research tools.

What you can do
  • Search the web for a topic and get back the most relevant passages from many sites in one reply
  • Read the full text of up to 20 pages or PDFs, or only the parts about a given topic
  • List the page addresses of a site and crawl many pages of one site, keeping the relevant ones
  • Run a multi-search investigation that returns a written report with numbered sources
  • Use the same search, extract, crawl and map requests and responses as Tavily
When to use it
Useful when an assistant needs current web answers with citations rather than raw links, and you want a hosted endpoint instead of running a search stack locally. Suited to research, pricing or documentation lookups and site-wide reading.
Requirements
Remote streamable HTTP endpoint; no local install. Requires an Apify API token sent as a Bearer value in the Authorization header. A free Apify account includes monthly credit; usage is billed per search and per page read.
Before you install
Paid per use: $4 per 1,000 searches and $1 per 1,000 pages read, drawn from your Apify credit. The Apify token is a secret and grants access to your account. Queries and fetched page content are sent to the hosted service and to Google. Research runs can take minutes.

Installation

In SourceWeft

  1. Open DeepSift Search in the dashboard and add it to a workspace.
  2. Connect Authorization token credentials. They are stored privately and never shown on public pages.
  3. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.pavanmulka%2Fdeepsift-search/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "deepsift-search": {
      "type": "http",
      "url": "https://aadstra--deepsift-search.apify.actor/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_TOKEN>"
      }
    }
  }
}

README

DeepSift

A free, self-hosted alternative to Tavily for Claude Code. Ask one question; DeepSift searches the web, reads every page, and hands back only the paragraphs that matter. It runs on your laptop and uses the Claude subscription you already pay for.

Tavily is a paid web search API built for AI agents. You send one question; it searches the web, opens the best pages, cuts out the passages that answer the question, and sends it all back in one tidy reply. That's great, but every call costs credits.

DeepSift does the same job on your own computer, in three steps:

  1. Find the right websites, using Claude's web search on your existing Claude subscription (with free DuckDuckGo as a backup).
  2. Read every page itself: HTML and PDFs, with menus, ads and clutter stripped out.
  3. Pick the passages that best match your question, and give them back in one reply.

No per-call bill, no API key to buy.

Don't want to run it yourself? Use DeepSift Search on Apify

[DeepSift Search icon]

DeepSift Search is the hosted version: real Google results, the same passage picking, nothing to install, and it works for apps and teams, not just your own laptop.

  • Tavily drop-in: same /search, /extract, /crawl and /map requests and responses. Change one line:

    python
    from tavily import TavilyClientclient = TavilyClient(api_key="<YOUR_APIFY_TOKEN>", api_base_url="https://aadstra--deepsift-search.apify.actor")

  • MCP server for Claude, Cursor and others: https://aadstra--deepsift-search.apify.actor/mcp with your Apify token as a Bearer header.

  • Claude Code plugin: /plugin marketplace add pavanmulka/deepsift, then /plugin install deepsift-search@deepsift. Claude Code asks for your Apify token once.

  • Pay per use, no subscription: $4 per 1,000 searches, $1 per 1,000 pages read. Failed requests are free, and a free Apify account includes $5 of credit every month.

The Apify version lives in .actor/ and deepsift/actor.py; it never uses your Claude subscription.

What Claude Code gets

After installing, every Claude Code session has these tools:

ToolWhat you'd askWhat it does
web_search"web_search Databricks Unity Catalog pricing"One topic → many websites → the best passages from each, in one reply
web_read"read these 3 pages and focus on pricing"Full text of up to 20 pages or PDFs, or only the parts about a topic
web_map"list all pages on docs.example.com"A site's page addresses
web_crawl"read the pricing pages on example.com"Reads many pages of one site, keeps the relevant ones
web_research"research the top 5 vector databases with sources"A multi-search investigation that returns a written report with numbered sources (takes minutes)

The same abilities work from any terminal:

sh
deepsift search "Databricks Unity Catalog pricing" --advanceddeepsift read https://example.com/page --focus "pricing"deepsift map https://docs.example.comdeepsift crawl https://docs.example.com --instructions "pricing"deepsift research "Which vector databases support hybrid search?"

Install (about 1 minute)

You need a Mac or Linux computer with Claude Code and Python 3.10+.

sh
git clone https://github.com/pavanmulka/deepsift.gitcd deepsift./install.shclaude auth login        # only if you aren't signed in to Claude Code's command line yet

Open a new Claude Code session and ask it to use web_search. That's it.

./uninstall.sh removes it from Claude Code again.

How it compares to Tavily

TavilyDeepSift
CostPay per creditFree (uses your Claude plan)
Speed per search1–2 s~10–30 s through Claude; ~1–3 s on the DuckDuckGo backup
Search, extract, crawl, map, research✓✓
Same request/response format—✓ (optional local HTTP API, see below)
Runs whereTheir serversYour laptop

Limits, honestly:

  • Searches through Claude count toward your Claude plan's usage limits (3 at a time at most). Repeat searches within 24 h come from a local cache.
  • DuckDuckGo pauses you after a burst of quick searches; the tool then rests it for 15 minutes.
  • Pages that only show their text after JavaScript runs come back thin.
  • It's meant for your own personal use with your own Claude subscription. To power an app for other people, use a proper paid API.

Optional: a local Tavily-compatible API

./run.sh starts an HTTP server on 127.0.0.1:8787 with Tavily's paths and fields (/search, /extract, /crawl, /map, /research, GET /research/{id}, /usage). Code written for Tavily can point its base URL there. Every reply reports usage.credits: 0.

Settings

Copy .env.example to .env to change engines, models, cache time, or set an API key for the HTTP server.

How it works

question ─► find sites (Claude WebSearch via `claude -p`, or DuckDuckGo)         ─► download pages in parallel (httpx) ─► readable text (trafilatura, pypdf)         ─► split into passages ─► rank by relevance to the question (BM25) ─► one reply

claude -p always runs on your subscription sign-in. ANTHROPIC_API_KEY is removed before it starts, so it never bills an API account by accident.

Tests

sh
.venv/bin/python -m pytest -q tests

The tests run offline, using a stand-in for claude.

License

MIT

Source: README.md at commit b37287a

Tools

0
This server asks for a credential such as an API key, so SourceWeft could not read its tools without one and has not indexed them.

Version history

1
  1. v0.1.0LatestNov 5, 2026

Score

85% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts