Darwin RAG

io.github.BrightdotdevPublished by GitHub account Brightdotdevv0.2.18Updated Oct 24, 2026

Local-first RAG engine with MCP server for AI agent integration.

The publisher's own server: Darwin

VerifiedSTDIODesktop onlyFiles & StorageKnowledge & MemoryData & Analytics

Overview

AI-generated overview

A local-first RAG engine that ingests documents, indexes them, and lets an assistant search and answer questions over them.

What you can do
Darwin RAG ingests PDFs, Markdown, HTML, images via OCR, CSV, Excel, ODS files and URLs, then builds a hybrid BM25 plus dense-embedding index with configurable chunking. Through MCP it exposes tools such as search_darwin for hybrid, semantic or keyword retrieval, search_lists and get_schema for structured files, run_pipeline to ingest and index, plus file status, saved-result and log queries. Answer generation is LLM-backed through LiteLLM, and multiple isolated stores separate tenants or projects.
When to use it
Worth adding when an assistant needs to answer questions over a private local document collection, including mixed formats such as scanned PDFs and spreadsheets, without sending the corpus to a hosted service. Also useful when you want separate knowledge stores per project and queryable session logs.
Requirements
Runs as a local process, installed from PyPI as darwin-rag, so Python is needed. Model setup is done first with the darwin-admin setup command. At least one LLM provider key is required for answer generation, supplied by environment variable such as OPENAI_API_KEY, ANTHROPIC_API_KEY or GEMINI_API_KEY; search and indexing work without one. DARWIN_BASE_DIR optionally overrides the data directory. Desktop only.
Before you install
The server asks for LLM provider API keys by environment variable, and those keys are used for answer generation, so queries and retrieved passages may be sent to the chosen provider. It can write to and delete local data: run_pipeline ingests and indexes files, and purge_artifacts deletes pipeline artifacts for specific files. HTTP or SSE mode can expose the server on a network endpoint, so bind it carefully.

Installation

In SourceWeft

  1. Open Darwin RAG in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

Darwin RAG

Search your own documents with any AI agent, privately, on your own machine.

Darwin is a local-first RAG engine that ingests documents, indexes them with BM25 + dense embeddings, and exposes search through an MCP server. Point Claude, Cursor, or any MCP client at it and your agent can search your PDFs, notes, spreadsheets, and web pages without sending them to a cloud index.

One-time internet is needed to download models during setup. After that, search and indexing run fully offline.

[License: MIT] [PyPI]

  • Ingestion: PDF, Markdown, HTML, images (OCR), CSV, Excel, ODS, URLs
  • Indexing: BM25 keyword + dense embedding hybrid index with configurable chunking strategies
  • Search: Hybrid, semantic, or keyword retrieval with reranking, diversity rerank, and structural penalties
  • Generation: LLM-backed answer synthesis via LiteLLM (OpenAI, Anthropic, Gemini, etc.)
  • Observability: Structured logging with per-session history, queryable via MCP
  • Isolation: Multiple independent stores for tenant/project separation
  • Deployment: stdio (AI agent subprocess), SSE, or Streamable HTTP; Docker-ready
  • Local-first: Everything runs locally, fully offline-capable after setup

Built by BrightDotDev.

Why Darwin?

  • Private by default. Your documents stay on your machine.
  • Works with your agent. Standard MCP server: Claude Desktop, Cursor, and other MCP clients.
  • Search that fits the question. Keyword for exact terms, semantic for meaning, hybrid when unsure.
  • Light to start. The required preset downloads just one embedding model.

Quick Start

1. Install

bash
pip install darwin-rag

2. Set up models

bash
# Interactive: detects hardware, pick your modelsdarwin-admin setup interactive
# Or one-shot (embedding-only, no prompts)darwin-admin setup --preset required

3. Start the MCP server & connect

bash
# stdio mode: for AI agent subprocess (Claude Desktop, Cursor, etc.)darwin mcp
# Or HTTP mode: for remote clientsdarwin mcp --http --port 8765

Configure your MCP client:

json
{  "mcpServers": {    "darwin": {      "command": "darwin",      "args": ["mcp"],      "env": {        "OPENAI_API_KEY": "sk-..."  // At least one LLM provider key      }    }  }}

Or generate config automatically:

bash
darwin config claude          # Claude Desktop configdarwin config cursor          # Cursor configdarwin config all --copy      # All clients + copy to clipboard

4. Try it

Ask your agent: "Ingest ./my-notes with Darwin, then find everything I wrote about ."

Use it as an agent skill

Darwin ships with a skill (skills/darwin-rag/SKILL.md) that tells agents when and how to use the MCP tools: check what's indexed, ingest, pick a search mode, and answer from sources. Copy the darwin-rag folder into your agent's skills directory (for Claude Code: ~/.claude/skills/).

MCP Tools

ToolDescription
search_darwinQuery the knowledge base with hybrid/semantic/keyword search
search_listsSearch structured data (CSVs, JSON arrays) by field values
get_search_resultsList saved search results
get_search_result_by_idLoad a saved search result by filename
get_schemaInspect schemas for structured files (keys, types, record counts)
run_pipelineIngest + index documents from a path or URL
purge_artifactsDelete pipeline artifacts for specific files
create_storeCreate a new isolated data store
list_filesList all tracked files with pipeline status
file_statusDetailed status for a single file across all stages
get_logsQuery session logs (oldest first, INFO excluded)

Full documentation: docs/mcp.md

Remote / HTTP Mode

Start the server on a network-accessible endpoint:

bash
# SSE transport (legacy)darwin mcp --sse --host 0.0.0.0 --port 8765
# Streamable HTTP transport (recommended for production)darwin mcp --http --host 0.0.0.0 --port 8765

Configure your MCP client with the URL:

json
{  "mcpServers": {    "darwin": {      "url": "http://your-host:8765/mcp"  // or /sse for SSE mode    }  }}

Environment Variables

VariableRequiredDescription
OPENAI_API_KEYNo*OpenAI provider key
ANTHROPIC_API_KEYNo*Anthropic provider key
GEMINI_API_KEYNo*Google Gemini provider key
MISTRAL_API_KEYNo*Mistral AI provider key
GROQ_API_KEYNo*Groq provider key
COHERE_API_KEYNo*Cohere provider key
TOGETHER_API_KEYNo*Together AI provider key
OPENROUTER_API_KEYNo*OpenRouter provider key
DEEPSEEK_API_KEYNo*DeepSeek provider key
DARWIN_BASE_DIRNoOverride the base data directory
NO_COLORNoSet to any value to disable ANSI color output

* At least one LLM provider key is required for answer generation. Search/indexing works without any.

Setup Details

CommandWhat it does
darwin-admin setup interactiveGuided setup: detect hardware, choose models
darwin-admin setup --preset requiredDownload embedding model only (fastest)
darwin-admin setup --preset recommendedEmbedding + reranker + OCR models
darwin-admin setup loggingReconfigure logging only
darwin-admin setup validateValidate current setup

See docs/setup.md for the full walkthrough including Docker, from-source install, and API key configuration.

CLI Reference

darwin: User CLI

CommandDescription
darwin mcpStart MCP server (stdio, --sse or --http for network)
darwin config [client]Generate MCP client config

darwin-admin: Power-user CLI

CommandDescription
darwin-admin setupSetup models, logging, and configuration
darwin-admin statusSystem status overview
darwin-admin modelsModel registry: list, install, switch, keys
darwin-admin storeData store: status, files, audit, health, repair
darwin-admin pipelineIngestion pipeline: run, ingest, index, purge
darwin-admin searchInteractive search
darwin-admin logsStructured log viewer and management
darwin-admin systemSystem information
darwin-admin uninstallRemove Darwin data and configuration

See docs/admin.md for the full command reference.

Python API

For embedding darwin-rag as a library in your own app:

python
from core import Darwin
d = Darwin()d.ingest("./papers", recursive=True)results = d.search("what is this paper about")records = d.search_records(filters={"status": "active"})

Full reference: docs/api.md

Documentation

DocWhat
setup.mdFull setup walkthrough
mcp.mdMCP server, tools, resources, transports
admin.mdAdmin CLI reference
architecture.mdFor developers and contributors
pipeline.mdIngestion & indexing
retrieval.mdSearch engine
storage.mdDarwinStore
models.mdModel registry & inference
logger.mdStructured logging
orchestrators.mdHigh-level business logic
api.mdPython API (Darwin class)

Contributing

Found a bug? Want to add something? You're welcome here.

Read CONTRIBUTING.md for the full guidelines.

License

MIT. See LICENSE.

Core architecture and implementation by BrightDotDev.

Source: README.md at commit dd74e2b

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.2.18LatestSep 16, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Darwin integrations

Darwin

so.darwinOfficial publisher

Darwin lets an assistant search for AI agents, inspect their operations, and coordinate multi-step work through them, including authentication and payment…

Streamable HTTPWeb executable
Productivity & Workflow15 toolsOct 26, 2026

Darwin-owned Public Sources

Published by GitHub account darwin-studios

Remote MCP server that lets an assistant fetch public HTML, Markdown, text or JSON documents from an approved list of official hosts, with provenance.

Streamable HTTPWeb executable
Web Search & Scraping2 toolsOct 31, 2026

Darwin-owned Files

Published by GitHub account darwin-studios

Free offline file conversions and bounded DOCX text extraction. Supplied bytes; no network.

Streamable HTTPWeb executable
Productivity & Workflow6 toolsNov 3, 2026

Darwin-owned AntV Charts

Published by GitHub account darwin-studios

Darwin-hosted AntV bar, column and line charts, plus bounded CSV bar charts. Free PNG output.

Streamable HTTPWeb executable
Data & Analytics4 toolsNov 3, 2026

Australia/Darwin clock

Published by GitHub account sadri-dridi

Remote MCP server whose headline tool returns the current local time in Australia/Darwin, bundled with assorted small time, URL, and validation helpers.

Streamable HTTPWeb executable
Location & Lifestyle31 toolsOct 27, 2026