io.github.Noshadi-sec由 GitHub 帳號 Noshadi-sec 發布v0.6.0更新於 Oct 29, 2026
Turn a research paper into working code and prove, claim by claim, how close it gets.
把 arXiv 論文變成可執行程式碼，實際跑起來，並針對每項論點給出可查核的證據紀錄與結論。
English · 简体中文 · 日本語 · 한국어 · Español · Italiano
Give it a paper. Get back working code and a report you can check without trusting the agent.
[PyPI version] [MCP tool catalog] [Python 3.11+] [MIT license]
Quick start · How IRP works · The report · What you can reproduce · FAQ · Documentation
uvx irp-agents setup
[!IMPORTANT] Get IRP from Noshadi-sec/irp or the irp-agents package on PyPI. Follow the installation and release links from these official sources. A similar repository name or copied README does not establish that a download is official.
Found a paper with no code, or code you cannot trust? Ask your agent to build it with IRP. The agent reads the paper, records every number it claims, writes the implementation, runs it and compares the results. IRP does the fetching, running and record keeping, and refuses the shortcuts: a number typed by hand never counts, a shortened run never counts, and the report is written only from the evidence log.
IRP connects your agent (Claude Code, Codex or Cursor) to tools for paper intake, environments, runs, measurements and verdicts. You can use the same tools from your terminal. Everything runs locally; IRP never calls a model, and data goes only where you send it.
With uv installed, run:
Choose your agents, review the proposed changes, and approve them. Setup adds IRP's MCP server and its workflow skill, with backups of existing configuration. Restart your agent afterward.
Setup supports Claude Code, Codex and Cursor. Other MCP clients can use manual registration.
You get back a working code folder plus a report: every claim, the paper's number, your number, and a verdict.
Every tool is also a command. Fetch a paper into a new study and read its outline:
To install the irp command for regular use:
See the CLI guide for the full workflow from the terminal.
Doctor checks Python, uv, git, Docker, GPUs, disk space and your agents' registrations, and changes nothing.
IRP changes quickly. For an installed command:
With uvx, refresh your agent setup with uvx irp-agents@latest setup, review the
plan, and restart your agent.
Your agent thinks; IRP acts and keeps the receipts. The agent reads the paper, writes code and decides what to run. Every action goes through one core, so the CLI and the MCP server behave the same, and every step lands in an append-only, hash-chained evidence log. The report is built from that log alone.
The workflow skill gives the agent six hard rules, and the tools enforce them wherever they can:
The report is the product: a person must be able to check every verdict without trusting the agent. It has five parts: the claim table, four verdicts (reproduced, close, not reproduced, could not run), the guess list, the method map from equations to code lines, and provenance (arXiv ID and version, file hashes, git commit, lock files, GPU, seeds, every command with its exit code, and a verify command that rechecks the hash chain).
The agent cannot finish until every claim has a verdict with evidence.
Example report.md (made-up numbers):
See evidence and report for every field.
IRP requires Python 3.11+ on macOS or Linux. Additional tools depend on the paper:
IRP supports computational papers: machine learning, algorithms and simulations. It does not support wet-lab work or anything weapons-related.
Five light papers — official code exists, each runs on a CPU in minutes — rebuilt
from the paper alone and checked claim by claim. Every number comes from a verified
evidence log; re-check any study with irp verify --study benchmarks/studies/<name>.
Across 8 claims: 6 reproduced, 1 close, 1 not reproduced (87.5% reproduced or
close), ~160s and $0 of paid compute per paper. Selection criteria and the full
set are in benchmarks/golden/papers.json; the
protocol and summarize.py are in benchmarks/.
If you build a paper with IRP, share the report in an issue with the paper, your question, and what you found.
Any agent that supports local MCP servers. Setup configures Claude Code, Codex and Cursor; other clients can use manual registration.
No. Your agent is the brain; IRP never calls a model and has no API key. It downloads papers from arXiv, installs packages through uv or Docker, and runs code on your machine. Your agent receives the tool results, and its model provider has its own data policy.
Running code is running code. A uv virtual environment is not a sandbox: the
code runs with your permissions. Use create_env with backend: "docker" for
code you did not write or do not trust, and runs mount only the study's code
directory. See security.
That this run, with the recorded guesses, did not reach the paper's number within the stated bar. The report says "not reproduced by this run" and lists every guess. It is evidence about one attempt, not a verdict on the paper; contact the authors before publishing it.
IRP builds missing code from the paper. To check that existing code reproduces its README's numbers, a tool such as paper-repro fits better. IRP is still useful for testing a paper by hiding its code and comparing your rebuild with it afterwards.
Only on paid GPU providers, only after you record a price and a budget at a
terminal with irp budget, and only after you approve the estimate the
agent shows you. The run's time limit equals the estimated hours.
Update first; a recent release may already fix it (irp update, or
uvx irp-agents@latest setup). If the problem persists, open an
issue with your IRP version,
the irp doctor --json output, the steps to reproduce and the error.
Report vulnerabilities through SECURITY.md.
See DISCLAIMER.md for responsible-use guidance. IRP is for lawful research, education and the verification of published computational work. A "not reproduced" verdict is evidence about one run, not a judgment of the paper or its authors.
Contributions are welcome. Open an issue to report a bug or suggest a feature, or send a pull request.
See CONTRIBUTING.md for development setup and checks, testing for verification lanes, and the architecture map for the project structure.
IRP-authored code is licensed under MIT. Third-party material retains its own licenses and notices; see Third-party notices. Papers and datasets keep their authors' licenses; IRP never redistributes them.
來源：README.md，提交 a64cab3
Fetch paper
Create the study directory and fetch the paper into it: arXiv LaTeX source first (exact equations), the PDF as fallback and for page images. Records arXiv ID and version, file hashes and an outline of sections, tables, figures, equations and algorithms. Call this first; a study holds one paper.
paper必填study必填parserpreferwith_pdfRead a section
Read part of the paper with its original file and line numbers: a section, a table, figure, equation or algorithm, a LaTeX label, the abstract, a PDF page, a line range, or 'index' for the full outline. Use the shown file:line when recording claims and method map entries.
id必填study必填offsetmax_linesLook at a page
Render one page of the paper PDF as a PNG image, for equations, figures and tables that text extraction loses. Saved under paper/pages/.
dpipage必填study必填Record a claim
Record one number the paper claims, before writing any code (rule 5): the metric, the value as printed, where it appears and the success bar. The value is checked against the paper text. Claims recorded after code exists are flagged in the report.
pagenotesstudy必填value必填where必填metric必填close_absclose_relstatement必填comparatorwhere_kindsource_file另有 4 個List claims
List every recorded claim with its success bar and current verdict.
study必填Record a guess
Record a detail the paper left out, the choice made and why (rule 2: never change the method to make a run succeed; record a guess instead). Every guess appears in the report.
why必填study必填choice必填claimssourcemissing必填Map method to code
Link an equation or algorithm step in the paper to the file and line in code/ that implements it. The file's hash is recorded so the report flags mappings whose code changed later.
notestudy必填end_linecode_file必填code_line必填paper_refpaper_element必填Create environment
Build a pinned environment for the study's code: a uv virtual environment (fast, not a sandbox) or a Docker image (isolated). Writes the lock file and records its hash. Requires recorded claims (rule 5).
gpunamestudy必填pythonbackendpackagesbase_imagerequirementssystem_packagesRun a command
Run one command in code/ with the named environment. Saves stdout and stderr, exit code, time, peak memory, seed, the code commit and the hash of every file the run writes. Only scope 'full' can count toward a verdict (rule 4). Use wait=false and run_status for long runs.
cwdenvnoteseedwaitscopestudy必填commitcommand必填networktimeout_smemory_limitRun status
Report a run's status, exit code and the last lines of its output; optionally wait for it to finish.
study必填run_id必填wait_stail_linesRun on a GPU cloud
Run one command on a paid GPU provider (Modal today; RunPod and Hugging Face Jobs planned). Returns a cost estimate and refuses until the person approves it (rule 6) and the estimate fits the budget a person set with `irp budget`. The time limit equals the estimated hours.
envgpu必填noteseedwaithours必填scopestudy必填command必填outputsapprovedprovider另有 1 個Extract metrics
Pull numbers from a completed run's stdout, stderr or the files it wrote, by key, regex, JSON pointer or CSV column. Each measurement keeps its file, line and file hash. Numbers from any other source are refused (rule 1).
specs必填study必填run_id必填Compare and judge
Put a claim beside its measurements and record one of four verdicts: reproduced, close, not reproduced, or could not run. Measurements must come from full-scope runs that exited 0; 'could not run' needs the blocking error (rule 3).
notestudy必填reasonclaim_id必填unit_scalecould_not_runfailed_run_idblocking_errormeasurement_idscontext_measurement_idsRecord a note
Record a limitation, a forced deviation from the paper, something not checked, or an author contact.
kind必填text必填study必填claimsWrite the report
Write report.md and report.json from the evidence log only: claim table, verdicts, guesses, method map and provenance. Refuses until every claim has a verdict with evidence (the completion gate) unless draft=true.
draftstudy必填bundleVerify the evidence
Recheck the evidence log's hash chain and the hash of every recorded file; reports each problem.
study必填Study status
Show the study at a glance: paper, claims and verdicts, guesses, environments, runs, and what still blocks a final report. Use it to resume work.
study必填已滿足適用檢查項的 90%
該評分衡量此收錄條目的完整程度，依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價，也不表示伺服器安全或目前可用：可連通性是帶日期的檢查結果，評分也不包含安裝量或使用者評價。
請在註冊中心條目中宣告憑證請求標頭，或保持端點能夠回應探測，以便判定其認證方式。
uvx irp-agents setupBuild arXiv 2501.12345 and check Table 2.irp fetch 2501.12345 --study ~/studies/exampleirp section index --study ~/studies/exampleuv tool install irp-agentsirp --helpirp doctorirp update# Reproduction report: Example Paper (arXiv 2501.12345v2)
Result: 2 reproduced, 1 close, 1 could not run (of 4 claims)
| Claim | Where | Paper | Ours | Verdict || --- | --- | --- | --- | --- || c1 Test accuracy | Table 2, row 3 | 94.1% | 93.8% (3 seeds, std 0.2) | close || c2 Training loss | Figure 4 | 0.31 | 0.31 (3 seeds) | reproduced || c3 Speed-up | Section 5.2 | 2.0x | 2.1x | reproduced || c4 ImageNet top-1 | Table 3 | 81.2% | not run: needs 8 GPUs for 2 days | could not run |
## Guesses- g1 Warmup length not stated: used 5 epochs. Why: as in cited paper [12].
## Method map- Eq. 3 (loss) -> code/src/loss.py:41- Algorithm 1, line 4 -> code/src/train.py:88
## Provenance- Paper: arXiv 2501.12345v2, source and PDF hashes- Code: git commit 3f2a9c1 (clean)- Commands logged: 37, 37 with an exit code- Evidence log: 112 records, head sha256:…, verified- Verify: `irp verify --study ~/studies/example`