forrt-research-mcp
[DOI] [PyPI]
An MCP server for producing verifiable FORRT nanopublication chains — the
tools a researcher needs while doing the work, not while looking for it. It
serves all three shapes of study on the same rails: reproduction, replication,
and research that starts from scratch.
It is the third of three servers that compose:
Built for the workflow in
forrt-replication-template,
but it needs nothing from that repo — any agent can call it.
Scope: reproduction, replication, and research from scratch
All three work here, and they use the same chain:
Step 01 is one of three anchors, and they are not variants of one form.
Which anchor fits is independent of whether the study is a reproduction, a
replication or original research:
The three share no field ids whatsoever, so call template_fields on the
anchor you are actually using rather than assuming they match. The
pico_question_type vocabulary is named for PICO deliberately: a PCC question
has no type to look up.
verify_quote applies to the Quote anchor only — and that is the one anchor
with a dedicated tool, because a quotation is the one field whose correctness
can be proved rather than reviewed.
constellation and prior_work surface a question anchor with its framework,
label, full question text and its framework's components — PICO's
population / intervention / comparator / outcome, PCC's population / concept /
context. Question nodes keep all of that in a question object and leave their
top-level label empty, so anything reading them like a quote returns nothing.
Tested against two real published question nanopubs.
For a study starting from scratch, the Claim is your own hypothesis, derived
from your own question or from the work you are building on. The FORRT Claim
template's source is optional, so it needs no external paper — and the
Claim-before-Study order then reads as
pre-registration, not as a mismatch. verify_chain takes a mode
(auto / replication / reproduction / new_research) that changes only what
is required: from-scratch research has no existing work to cite, so no CiTO
step and no cited DOI are expected.
The one place the templates still assume an original is the study_type
vocabulary on 04_study, whose three terms are all replication-flavoured
(Replication Study, Reproduction Study, or both). There is no term for an
original study testing its own claim. Three field prompts read oddly too —
scope and methodology say "is reproduced/replicated", and the Outcome's
conclusion says "about the original claim" — but they are only wording;
validationStatus (validated / partially supported / contradicted /
inconclusive / not tested) describes testing your own hypothesis perfectly well.
So closing the gap is plausibly one added vocabulary term and three reworded
prompts, not a new template family. When that lands, this server picks it up
with no code change: template_fields and vocabulary fetch live, and
driftedFromSnapshot flags the supersession so the vendored copy gets re-cut.
A note on using "Reproduction Study" for from-scratch work. It is a
reasonable workaround while the vocabulary lacks a better term, but the
template means the replication-science sense — "direct reproduction: same
methodology, same tools", i.e. re-running someone else's analysis — not the
RSE sense of "my work is reproducible". Downstream consumers read it the first
way: replication-radar's verdict overlay and verified_claims will present
the study as verification of an existing claim.
Why a server and not a prompt
Two jobs here look like reasoning but are not, and doing them in an agent loop
makes them unreproducible and expensive:
1. Reading a published chain. /np/constellation walks the FORRT citation
graph bidirectionally and returns everything it reaches. For a single
marine-heatwave chain that is 330 KB, 98 nodes and 1012 edges — of which 64
nodes are AIDA statements belonging to entirely different studies. Handing
that to an agent burns its context and invites it to reason over another paper's
claims. constellation returns the same chain in ~16 KB (4.7 %).
2. Checking a quotation is real. A FORRT Quote nanopub must be verbatim. That
is a string search, not a judgement — so it should be a tool that cannot be
talked out of its answer, and that anyone can re-run to get the same result.
What it does not do
It does not search papers (that is the OpenAIRE MCP), rank replication targets
(that is replication-radar), or write nanopub content. It also does not
extract claims: choosing which sentence carries a paper's headline claim is a
judgement, and wrapping a judgement in a tool would not make it reproducible — a
model behind a tool boundary is exactly as non-deterministic as one in the agent
loop. The division this server is built around:
Install
User-scoped (-s user) so it is available in every session and folder, and does
not clash with a per-repo config. For other agents, the stdio command is
forrt-research-mcp.
The default base is production. To test against the dev deployment, set
SCIENCELIVE_API_BASE=https://api-dev.sciencelive4all.org. A constellation
that nobody has requested recently can take ~60 s to build; after that it is
cached for 4 h and returns in well under a second.
Tools
verify_quote(pdf_path, quotation)
Proves a candidate quotation is in a source PDF before it is published as
verbatim. Returns a graded verdict with page, character offsets and the file's
SHA-256:
Every tier canonicalises formatting only — never words, digits or order. A
quotation with one digit changed scores ~0.91 similarity against the source and
is still not_found; on a miss, closest.text_in_pdf shows what the paper
actually says there.
A tier below exact is normal. PDF extraction inserts line breaks, loses
hyphens, and sometimes breaks words apart entirely:
- the quotation published in the real marine-heatwave chain matches only at
extraction_tolerant, because pypdf reads the paper's 35-year as 35year
and (p < as ( p <;
- Hundhausen et al. 2024 renders its own abstract as
CP en semble, largest c hanges, prec ipitation — spaces inside words, which no punctuation rule
can undo. That needs whitespace_insensitive, which deletes every space and
therefore does not verify word boundaries; it declines to run below 40
characters, where two short texts could collide.
"Character-for-character" is not literally achievable against extracted PDF
text, which is why the result is graded rather than boolean.
Words split across a line break (convection-\npermitting) are rejoined at the
normalized tier. A suspended hyphen (15- and 30-minute) is deliberately
NOT rejoined — the rule requires a line break, because joining those would
corrupt the text.
constellation(uri, depth=5, max_nodes=80)
The FORRT chain(s) reachable from a published nanopub URI, projected to
agent-size: chains with their steps in chain order, the apex CiTO, any Research
Synthesis, and the Quote/Question anchors attributable to this paper.
Read citedPaper, not the API's top-level paperDoi — see Upstream
quirks below.
stepsPresent legitimately omits steps: a CiTO at the apex of a constellation
is hoisted out of its chain, and Quote/AIDA anchors are often not enumerated.
Missing does not mean unpublished.
prior_work(uri)
What has already been claimed about a paper — the starting point for new work,
replication or otherwise. Per completed chain: claim type, scope, method,
deviations, verdict, confidence, repository, and limitations.
limitations is the field to read most carefully. It is where previous authors
stated, in signed and immutable words, what their study did not cover — which
is often exactly where the next study begins. In the marine-heatwave chain it
records that the paper's better-known 54 % headline describes a different
analysis over a different period and was not tested.
constellation_raw(uri, …)
The unprojected response, for debugging the graph or an upstream data problem.
Large; prefer constellation.
template_fields(step, live=True) · vocabulary(name, live=True) · list_schemas()
A nanopub template is the schema for its chain step, so these turn "never
invent a field name" and "never invent a claim type" from rules an agent has to
remember into a lookup that can only return real values.
template_fields returns the real field ids, prompts, required/repeatable
flags, and the regex / prefix / datatype constraints — which is where the
Quote template's character cap actually lives, rather than in a hand-written
doc that drifts. step accepts 05_outcome, 05, or outcome.
vocabulary returns the allowed values of a controlled vocabulary, taken from
the real restricted-choice field:
Both fetch live by default and fall back to a bundled snapshot when the
network is unavailable — always reporting which was used, and setting
driftedFromSnapshot when the upstream template has been superseded. A drifted
result means the live values win and this package needs re-vendoring; it is a
loud, reviewable event rather than slow silent divergence.
cito_relation is the one vocabulary that cannot be resolved offline: it lives
in a separate value-list nanopub. Offline it returns source: "unavailable-offline" with a warning rather than an empty list, because an
empty list reads as "no valid values."
verify_chain(published_path, repo_url="")
The final check before announcing a chain. Point it at a nanopubs/PUBLISHED.md
ledger; green is true only when nothing failed. Read-only — it never edits,
retracts or supersedes.
That last one is the failure most worth catching: Validated implies
confirms, PartiallySupported implies qualifies, Contradicted implies
disputes. A mismatch means the Outcome and the Citation disagree about what
the replication actually found — a chain that cites confirms for a paper it
contradicted is worse than no chain at all.
Only the ledger's own Outcome and CiTO are judged. Published work is a
network: chains citing the same paper turn up in the same walk, and one CiTO
cites several works (confirms the paper, usesMethodIn a method,
citesAsDataSource data). So the relation→target pairs are read from the
ledger's step-06 TriG, and the verdict is checked against the relation on the
paper the Quote (step 01) quotes. Other chains are listed as neighbours. If the
check cannot be made, it says so with a warn rather than skipping it.
A constellation alone cannot verify a chain, which is why this does not try.
On both real chains the walk stops short of the upstream anchors, so anything it
does not enumerate is checked by fetching its TriG from the bare
w3id.org/np/ resolver — the /sciencelive/np/ form serves an HTML viewer that
answers 200 and would pass a status-only check while serving no nanopub. A row
reading "not enumerated by the walk but its TriG resolves" is a pass, not a
warning.
The Outcome's repository is expected to be a Zenodo version DOI, not a
GitHub URL: a version DOI pins the archived state the outcome was computed
from, where github.com/ORG/REPO is a moving target. Both are accepted; a DOI
is checked by resolving it, a URL by comparing it to repo_url.
Both published chains verify green, matching the hand-run verification recorded
in marine-heatwave's ledger.
validate_chain_draft(path)
Checks nanopubs/chain-draft.json — the artifact that actually gets
published — before it is handed to the Science Live chain wizard. Run it at the
end of Phase 5b, after pixi run build-chain-draft and before pushing the file
and opening the wizard URL.
The markdown drafts are the authoring format; build_chain_draft.py turns them
plus CITATION.cff and the templates into this file, which the wizard pre-fills
each step from and a human reviews and signs. validate_draft checks the input;
this checks the artifact.
What it catches that reading the file cannot:
- a superseded
template_uri — invisible in the JSON, but it makes the
wizard pre-fill the old form;
- a
prefill key that is neither a template field nor a known platform
form-field, which the wizard silently drops;
- a complex field in the wrong shape —
06_citation.st02 must be
[{cites, cited}] with at least one entry, and 04_study.disciplineSelection
is a single object, not an array (the one asymmetry in the contract);
- a required field neither prefilled nor carried forward;
- a value over the template's own cap, an invalid vocabulary term, a malformed
date, an unresolved
{{TOKEN}}, or a DOI that does not resolve;
- a
carry_forward edge running backwards through the chain.
Fields the wizard fills itself are exempt rather than reported missing:
02_aida has no project, 03_claim no aida, 04_study no claim.
Both of this project's real chain drafts validate clean, and 13 deliberate
mutations of one are each caught.
validate_draft(path) · validate_drafts(directory)
The pre-flight checklist in docs/forrt-form-fields.md, actually executed. One
call checks a drafted nanopub end to end: field ids against the live template,
choice values against its enumeration, length caps against its regex, DOIs
against the registry, QIDs against Wikidata. publishable is true only when
nothing came back as an error.
This is the composition the other tools exist for — individually they answer
"is this value real?", together they answer "is this draft publishable, and if
not, exactly where is it wrong?"
Three placeholder conventions, and only one is a problem. Getting this wrong
made the checker useless on real drafts, so each is handled explicitly:
A draft whose required fields are nearly all empty is reported once as an
unfilled skeleton — "it has not been drafted yet" — rather than as a wall of
per-field errors.
What it cannot see. Values a draft puts in prose or a markdown table rather
than behind a <!-- field: … --> marker are reported as coverage warnings,
never as missing. The CiTO step's citation list is the known case: cites and
cited live in a table in both repos checked.
Validated against two real published chains (marine-heatwave and Sado estuary):
all six published steps pass in both, with the only remaining flags being an
unpublished step 07 and the release tokens above.
resolve_doi(doi)
Does this DOI resolve, and to what? resolves: false means it is not
registered — a well-formed DOI is not a real one, and a fabricated one is
indistinguishable from a genuine one until something asks the registry. Returns
the registered title, authors, year and container so you can confirm it is the
paper you meant, not merely a paper that exists. Accepts bare, URL, and doi:
forms. A 5xx is reported as transient rather than as a bad DOI.
wikidata_lookup(query, expected_type="", limit=5)
Real Wikidata candidates for a term, with their actual P31/P279 types. Pass
expected_type as a QID and each candidate is marked typeMatches.
Checked against the ten Wikidata terms in this project's published chains — the
QIDs build_chain_draft.py resolved and signed — it returns the published QID
at rank 1 for all ten (marine heatwave, sea surface temperature, climate change,
time series analysis, chlorophyll a, remote sensing, estuary, Sentinel-2, water
quality, atmospheric correction).
It deliberately does not choose — picking the right sense of an ambiguous
label is a judgement. Searching Bombus with Q16521 (taxon) returns the
insect genus as a match and the album of the same name as not, which is
exactly the failure worth catching before a QID gets signed into a nanopub.
Candidates are annotated, never filtered: a near miss is often the informative
result. Zero candidates means leave the field empty — never fall back to a QID
from memory.
cito_to_wikidata(cito_uris=None, published_path="")
Makes a published typed citation ("this replication confirms that paper")
visible in Wikidata and Scholia. Wikidata already models CiTO: "cites work"
(P2860) with the qualifier "has goal" (P3712) set to one of 48 citation-intent
items, each of which has an exact match (P2888) to its CiTO URI — so the
mapping is read from Wikidata at run time, never copied here (a copy would go
stale without saying so; if Wikidata cannot be reached the tool says so). The
statement is referenced with "nanopublication identifier" (P12545) in the
https://w3id.org/np/<id> form, which Scholia counts as an annotation defined
in a nanopublication.
It never edits Wikidata. It returns a QuickStatements batch and a per-work
report; a person reviews them and submits the batch at
https://quickstatements.toolforge.org with their own account. Also available as
a command:
What it guarantees:
- items are looked up by DOI (and authors by ORCID) and created only when missing;
- one item per work: Zenodo versions are grouped under their concept DOI, so
two versions of one deposit do not become two replications;
- idempotent: citations already in Wikidata are skipped; a citation to a work
created by the same batch is listed as
pending and added on the next run;
- every fact on a new item (type, title, DOI, date, authors) cites the Zenodo
record it was read from: Wikidata is a secondary knowledge base, and the citation
itself cites its nanopublication;
- the description is short prose ("reproduction or replication study by ()"), keeping identifiers in statements where they belong.
Items are created from the Zenodo record's title: fix a repository-slug title on
Zenodo before submitting.
validate_rocrate(path, profile="ro-crate-1.2", level="REQUIRED", metadata_only=False, offline=False)
Checks a replication's ro-crate-metadata.json before it is released or
deposited in ROHub, with the same validator the template's CI runs
(rocrate-validator). path is the
crate's directory or its metadata file.
passed / issues — the specification check against profile at level
(REQUIRED, RECOMMENDED, OPTIONAL); each issue names the check, its message and
the offending entity. metadata_only=True skips the checks that listed files
exist, for a crate whose data are not checked out. The first run fetches the
RO-Crate JSON-LD context and caches it; offline=True relies on that cache.fdo_checks / fdo_ready — what a crate needs beyond the specification to
work as a FAIR Digital Object: a persistent identifier, a licence, the
workflow as mainEntity, a community FAIR Implementation Profile in
conformsTo, the FORRT chain in subjectOf, and no ro-crate-metadata.json
among its parts (ROHub reserves that name). Advisory — they never change
passed — and each failing check says how to fix it.
The crate is generated by the template's scripts/build_rocrate.py: fix its
sources (CITATION.cff, nanopubs/PUBLISHED.md, rocrate.json) and rebuild,
never hand-edit the file to make it pass.
Upstream quirks this handles
Found by probing live data rather than reading the API contract, and each one is
pinned by a test against a real recorded payload (two constellations, from
unrelated studies):
- Cloudflare rejects urllib's default User-Agent. The API runs on
Cloudflare Workers, whose bot protection answers
Python-urllib/3.x with
HTTP 403 — same URI, same key, 200 under any normal agent. Every request this
client makes sets an explicit User-Agent. Do the same in your own client;
no hermetic test can catch it.
paperDoi can name the wrong paper. It is chosen by frequency across the
whole walk, so a neighbouring study's Quote nanopubs can outvote the chain's
own citation. On the marine-heatwave chain it reports the LifeWatch ERIC
paper (4 unrelated quotes) instead of Oliver et al. 2018, which the chain's
CiTO actually cites — and it names the same wrong paper from the
unrelated Sado-estuary entry point, so this is systemic, not one bad record.
citedPaper derives the paper from the CiTO and sets
disagreesWithReported when the two differ.- The walk stops short, and missing does not mean unpublished. Two real
chains enumerated
[Claim, Study, Outcome] and [Study, Outcome, CiTO],
while the Quote, AIDA and Claim nanopubs listed in those repos'
PUBLISHED.md were published and merely unreachable. stepsNotEnumerated
reports the gap. A constellation alone cannot verify a complete chain —
fetch those URIs' TriG directly.
quote.quotedText comes back empty on nanopubs using the current quote
template. The text survives in label and in two untagged excerpts — the
quotation and the annotator's comment, in no guaranteed order. We match them
against the label stem rather than assuming a position, and report the
recovery in text_source.- The apex CiTO is hoisted out of
chains[].steps[] to top level.
- The
/sciencelive/np/ URI form serves an HTML viewer that answers 200, so
a status-only reachability check passes even when no nanopub is served. We
normalise to the bare w3id.org/np/ resolver and assert the body is RDF.
Checking against every real artefact
The test suite is hermetic: it pins behaviour against recorded data, which
prevents regressions but cannot discover a wrong premise. Every serious bug
in this server was a wrong premise, and each surfaced only when the tools met
real data they had not seen. So "have we tested comprehensively?" is not a
judgement call here — it is a computation with a visible denominator:
It discovers every repository with a nanopubs/ directory, runs the applicable
tools against each artefact, and prints what passed out of what was found.
Findings are not automatically failures — an empty required field is a true
statement about a draft. What it exits non-zero for is a crash or a parse
failure, because those mean a tool could not read a real artefact at all.
The denominator is only as good as what is on disk. A chain that is not under
that root is not covered, and a green run does not say otherwise.
Development
All 287 tests are hermetic and need no network, no API key, and no live
service: the constellation fixtures are real recorded /np/constellation
responses, quote tests build minimal PDFs in-process that reproduce the
extraction artifacts deliberately, and the template/DOI/Wikidata tests stub HTTP
with recorded payloads. The suite stays green through an upstream outage.
The bundled template snapshot under src/forrt_research_mcp/data/ is vendored
from ScienceLiveHub/forrt-replication-template, as is template_spec.py.
Re-vendor when template_fields reports driftedFromSnapshot.
License
MIT — see LICENSE.
