Thoth

io.github.ahmedEid1Published by GitHub account ahmedEid1v1.0.1Updated Oct 24, 2026

Read-only MCP over an agentic SLR workspace with per-claim citation verification

VerifiedStreamable HTTPWeb executableWeb Search & ScrapingKnowledge & Memory

Overview

AI-generated overview

Read-only MCP access to Thoth's agentic systematic literature review workspace, including per-claim citation verification reports.

What you can do
  • List your systematic literature reviews with critic and citation-faithfulness scores
  • Read the markdown draft of a completed review
  • Retrieve the per-claim citation audit that labels each cited claim supported, unsupported or unclear
  • Inspect papers surfaced by the discovery step, with fetch and screening status (v2)
  • See the search queries the discoverer generated and any per-provider errors (v2)
When to use it
Useful when you already run a Thoth instance and want an assistant to pull up finished literature reviews, drafts and citation audits, or to check whether a draft's citations actually hold up against the source papers before you read it.
Requirements
A running Thoth deployment: the hosted app was retired, so you self-host it (Node.js/pnpm, Docker Compose for Postgres, MinIO and Langfuse, Prisma migrations, a Trigger.dev worker) and point the client at your own MCP URL. OAuth 2.1 with PKCE and dynamic client registration via Clerk runs in the browser. Self-hosting needs Clerk and Trigger.dev keys plus an LLM provider key such as MISTRAL_API_KEY.
Before you install
The public hosted endpoint is retired, so the registry URL will not work; you must deploy and secure your own instance. Tools are read-only and scoped to your account, but they expose review drafts, citation audits and discovered papers, and the audit log stores a SHA-256 hash of each call's input. Self-hosting requires third-party credentials (Clerk, Trigger.dev, LLM provider) and outbound calls to OpenAlex, arXiv and Exa. The README reports low claim faithfulness (38%) and uncalibrated v2…

Installation

In SourceWeft

  1. Open Thoth in the dashboard and add it to a workspace.
  2. Connect the required credentials. They are stored privately and never shown on public pages.
  3. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/io.github.ahmedEid1%2Fthoth/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config. The client completes OAuth sign-in on first connect.

{
  "mcpServers": {
    "thoth": {
      "type": "http",
      "url": "https://thoth-slr.vercel.app/api/mcp/mcp"
    }
  }
}

README

[Thoth — sacred ibis logo]

Thoth

Agentic systematic literature reviews — with every citation checked against the source.

Named for Thoth, ancient Egypt's ibis-headed god of writing and scribes.

[Hosted app] [Tests] [App release] [Deploy cost] [License: MIT]

The hosted app ran live in 2026 and is now retired. The screenshots and walkthroughs below are from it; you can still run it yourself.

[Browsing a completed Thoth review — draft, critic score, and per-claim citation audit]

What is Thoth?

Systematic literature reviews are slow to write — and when you ask an LLM to write one, it confidently invents citations and statistics that aren't in any paper.

Thoth does both halves and checks its own work. Give it a research question and it discovers relevant papers, reads them, drafts an evidence-grounded review — then runs a verification pass (cite_check) that compares every cited claim against the source paper and flags anything unsupported before you read the draft. The result is a review with a critic score, a citation-faithfulness percentage, and a per-claim audit you can trust.

It ran (hosted, in 2026) as a polished web app, a public eval dashboard, and an authenticated MCP server your AI assistant could call directly; it still runs on your own machine or host.

See it work

Claude.ai catches 6 fabricated citations in a real draft — using Thoth's audit:

Connected to Thoth via the official MCP Registry, Claude calls get_citation_audit on one deliberately-weak review (faithfulness 0.13 for that single review) and identifies all 6 unsupported claims — every one citing the same paper, with invented percentages that aren't in the source. This is cite_check doing its job: it's a single-review audit sample, not the golden-set aggregate (the /evals dashboard had that).

Every claim, scored against its source — the /showcase review (no login needed). The figures on this card (critic 4.2/5, faithfulness 75%, 8/8 citations checked, 2 unsupported) are this one review's scores — a worked example, not the aggregate:

Evaluated in public — the /evals dashboard tracked inclusion recall / precision, claim faithfulness and coverage over an 18-question versioned golden set (7 of 18 have data at the latest sweep's commit), regenerated in CI and published with the last-run date (last run 6 July 2026, before the hosted app was retired), so a regression was a public, falsifiable signal:

You approve every step — three human-in-the-loop gates (review plan → review discovered papers → approve included papers); nothing runs unattended:

Key features

  • 🔎 cite_check — verifiable citations. Every [paper_id] in the draft is scored against the cited paper and labelled supported / unsupported / unclear, so the LLM can't quietly hallucinate a citation. The verdict report is published per claim, not summarised away (claim faithfulness on the golden set: 38%). This is the core differentiator: the citations are measured, not asserted. Paper selection is scored separately: the papers Thoth includes are compared with each golden question's expected paper list — inclusion precision 97%, recall 74%, averaged over the 9 golden questions that have an inclusion score. That measures which papers it picks, not whether its citations are correct, and precision is 1.0 by construction on goldens with no distractor papers (see known limitations).
  • 🌐 Outbound web search (v2 — under active evaluation). An outbound discoverer → fetcher → screener path is wired across OpenAlex, arXiv, and Exa: it fetches open-access PDFs, OCRs them, and screens each against your plan, so you can run uploaded-only, hybrid, or fully autonomous discovery. The discovery and screening axes are v2 and still being calibrated — they're tracked openly on /evals (both at 0% at the last run) rather than shipped as a silent claim.
  • 🔌 Authenticated, registered MCP server. OAuth 2.1 + PKCE + Dynamic Client Registration via Clerk, SHA-256 audit logging, rate limits — listed in the official MCP Registry. Most public MCP servers ship with no auth; this one doesn't.
  • 📊 Public eval dashboard. Inclusion recall / precision, faithfulness and coverage over a versioned golden set, regenerated in CI and stamped with the last-run date, rendered at /evals — an eval regression was a public signal, not a hidden one. The weekly eval run is retired; the last run was 6 July 2026.
  • 💸 6 LLM providers, $0/mo by default. Swap providers with one env var; the hosted app ran on the Mistral free tier, and the entire stack deploys on free tiers for $0/mo.

🚀 Quickstart

The hosted app ran live in 2026 and is retired, so run your own instance (below).

Connect it to your AI assistant — paste your deployment's MCP URL into claude.ai (Pro/Max), Claude Desktop, Cursor, or any MCP client (OAuth runs in your browser; no token to copy):

https://<your-host>/api/mcp/mcp
Read-only MCP tools (scoped to your account)
  • list_reviews — your reviews with critic + faithfulness scores
  • get_review_draft — the markdown draft of a completed review
  • get_citation_audit — the per-claim cite_check verdict report
  • list_discovered_papers (v2) — papers the discoverer surfaced, with fetch + screening status
  • get_search_queries (v2) — the queries the discoverer generated + per-provider errors

Full reference: docs/mcp/tools.md · auth + audit model: docs/mcp/security.md

[Adding Thoth as a custom MCP connector in claude.ai — paste the URL, OAuth via Clerk + Dynamic Client Registration]
Adding Thoth as a custom connector in claude.ai — OAuth runs in your browser (Clerk + DCR), no token to copy.

Run it locally:

bash
git clone https://github.com/ahmedEid1/thoth.git && cd thothcp .env.example .env        # Clerk + Trigger.dev keys + MISTRAL_API_KEYdocker compose up -d        # postgres, minio, langfusepnpm install && pnpm prisma migrate devpnpm dev                    # Next.js on :3000pnpm dev:trigger            # Trigger.dev worker (separate terminal)

Full setup, the agent pipeline, and the v2 flow: docs/architecture.md.

Proof

Hosted appRan live in 2026 (Clerk sign-in, sample review at /showcase); retired on 1 Nov 2026
Public evals/evals (last run 6 July 2026) — inclusion precision 97%, recall 74% (papers included vs. each question's expected list, averaged over the 9 golden questions with an inclusion score) on a versioned 18-question golden set (7 of 18 have data at the latest sweep's commit; faithfulness 38% / coverage 32% tracked in the open as the set fills out; discovery/screening v2 under calibration). Regenerated in CI, published with the last-run date — a regression is a public signal.
MCP Registryio.github.ahmedEid1/thoth — listed; its remote endpoint was retired with the hosted app
Tests676 unit/integration + 22 live e2e against the deployed instance (MCP transport, real-browser, authenticated walkthroughs, full agent runs) — all green; tsc + lint clean
Audit logEvery MCP call recorded with a SHA-256 input hash; no raw input stored
Deploy cost$0/mo — Vercel + Neon + Cloudflare R2 + Langfuse + Trigger.dev, all free tiers (self-host option)

For engineers

Thoth is a LangGraph StateGraph driven by a Trigger.dev worker, with durable human-in-the-loop gates, a per-run cost cap, and exactly-once gate delivery. Next.js 16

  • TypeScript (strict), Postgres + Prisma, Clerk auth (web + OAuth 2.1 for MCP), S3-compatible storage, Mistral OCR, Langfuse tracing.

Credits

Ibis icon by Delapouite under CC BY 3.0, via game-icons.net.

License

MIT © 2026 Ahmed Hobeishy

Source: README.md at commit d7c5b4b

Tools

0
This server asks you to sign in, so SourceWeft could not read its tools without an account and has not indexed them. What a connected account sees can depend on that account and its permissions.

Version history

1
  1. v1.0.1LatestSep 16, 2026

Score

60% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declaredRequired10 pts
  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts