TheCrawler

io.github.manchittlabv0.5.8Updated Sep 23, 2026

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

VerifiedSTDIODesktop onlyFiles & StorageBusiness & CommerceKnowledge & Memory

Installation

In SourceWeft

  1. Open TheCrawler in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

Tools

0
Tool metadata has not been indexed yet.

Version history

1
  1. v0.5.8LatestSep 16, 2026