Monte-Neo strategy verifier

[Monte-Neo logo]

[Monte-Neo: the independent verifier for trading strategies written by AI agents and humans]

The independent verifier for trading strategies written by AI agents and humans.
Catch look-ahead bias, hidden trading costs and overfitting before a backtest reaches your money.

[PyPI version] [Total downloads] [Downloads per month] [Python versions] [CI] [MIT license]
[MCP Registry] [Tested with Claude Code and Qwen Code (local)] [GitHub stars]

Docs · Quick start · Use from agents · Verifier API · Trap Suite · Changelog

Did it catch a leak in your backtest? Star the repository so other quants and agent builders can find it, or tell us what it missed.

The problem

A coding agent can turn a trading idea into a backtest in minutes. It will then tell you the strategy returns 40% a year with a Sharpe of 3. Most of the time that number is wrong, for the same few reasons:

• Look-ahead bias. The code reads future bars: shift(-1), centred windows, bfill, statistics over the whole series, np.gradient, an FFT filter.
• Missing costs. The edge is smaller than fees and slippage, or it disappears when the fill comes one bar later.
• Selection bias. The agent tried 300 variants and reports the best one as if it were the only one.

Backtest libraries run whatever code you give them. None of them tell you the backtest itself is broken.

The solution

Monte-Neo checks the backtest, not the idea. Give it the price data and the strategy code or its positions. It returns one of four verdicts, the checks behind the verdict, concrete next steps and a reproducible, optionally signed certificate.

[monte-neo verify rejects a leaky agent strategy, the agent fixes it, and the honest verdict follows]

The agent's strategy used shift(-1), so it knew the next close. Monte-Neo found the leak in four independent ways and named the line. After the fix, no look-ahead is left, and the verifier tells the truth: on a random walk, the strategy has no edge after costs.

The same run as a report for people (--html report.html: one file, no scripts, no network):

[HTML report of a rejected strategy: the verdict in one sentence, a card per check family, the summary, the equity curve against buy and hold, the drawdown and the return distribution]

More reports, including an honest strategy and a parameter search, are in the gallery.

$ monte-neo verify --ohlcv prices.csv --strategy agent_strategy.py --n-trials 40REJECT  certificate 7f0603b0e604bea5  check                    category    status  summary  data_integrity           integrity   pass    OHLCV is clean  lookahead_truncation     lookahead   fail    truncation probe: LEAK DETECTED  lookahead_perturbation   lookahead   fail    future-perturbation probe: LEAK DETECTED  lookahead_static_lint    lookahead   fail    static lint: negative_shift  implausible_accuracy     lookahead   fail    next-bar hit rate 1.000 over 2999 bars (z 54.8)  net_profitability        economics   fail    net total return -64.97% after costs  deflated_sharpe          statistics  fail    deflated Sharpe 0.000 over 40 trial(s)  ...                                          (16 more checks)→ The signal at bar t changes when later bars are removed: compute features only from rows <= t  (no shift(-k), centered windows, bfill or full-sample stats).→ Fix the flagged source lines (negative shift, center=True, backward fill) and re-run verify. Lines: 6.

New in v0.53: from a rejected backtest to a committee

Repair the strategy, without overfitting it

[monte-neo repair: a leak is repaired and no edge is left; a noisy trend signal is smoothed and still earns on a lockbox opened once; an unseeded strategy is not repairable]

monte-neo repair --demomonte-neo repair --ohlcv data.csv --strategy my_strategy.py --out repair/

Optimisers maximise the return, so they overfit. repair minimises defects instead. It first corrects look-ahead and repainting. Then it picks from a short list of declared repairs (smoothing, holding period, no-trade band, tranches, neutralisation, caps, volatility target) on the first 70 % of the history, by the number of defects left and never by return, and counts every candidate as a trial. The last 30 % is a lockbox, opened once.

There are four outcomes:

• REPAIRED: on data the choice never saw, the strategy still earns after costs and is not rejected (the lockbox can still ask for more evidence, such as more trades).
• REPAIRED_NO_EDGE: the defects are gone and so is the profit, because the profit was in the defect.
• NOT_REPAIRABLE: what you have to do yourself.
• NOTHING_TO_REPAIR: there was nothing in the catalogue to fix.

The original file is never touched. On a random walk, none of the 68 leaking strategies of the Trap Suite comes out as REPAIRED. Guide.

Is it alpha? An Alpha Readiness Level for the committee

monte-neo alpha --demo --aum 2e5monte-neo alpha --ohlcv universe.csv --scores scores.csv --aum 5e7 --book book.csv --n-trials 40

A ranking signal gets the checks a fund's research committee runs before it gives capital:

• signal power: rank IC, decay, quantiles, placebo scores;
• attribution: is it alpha, or momentum and short volatility under a new name?
• value to your book: the marginal Sharpe;
• implementability: square-root impact at your capital, capacity, short leg, rebalance day;
• robustness: subsets, start dates, bootstrapped markets;
• data and process: today's index members, mismatched exchange closes, decay after publication;
• multiple testing: the t ≥ 3 hurdle, the minimum backtest length;
• risk: named crises, tails.

Everything ends in one line, for example ARL 6/9 — blocked at 7: tradable and robust (rebalance_timing_luck). Missing evidence is "not assessed", never passed. Guide.

Evidence a model-risk committee can file

monte-neo dossier --certificate cert.json --framework sr11-7 --name "Momentum 20" --owner "Desk A"monte-neo monitor --certificate cert.json --live live.csv        # after launch: is the alpha dead?

A validation dossier is one folder with:

• one printable page: summary, checks, readiness level, assumptions and limitations, how to reproduce, sign-off fields;
• a clause map to SR 11-7, PRA SS1/23, MiFID II RTS 6, a crypto checklist or an academic appendix;
• the certificate and the environment.

The map shows evidence, never compliance. monitor compares live returns with the backtest, measures realised slippage and raises sequential alarms when the edge dies. Guide.

Machine learning, LLM signals, events and track records

• fit / predict. Define fit(train) and predict(model, df) instead of signal(df). The verifier refits only on the past, and ml_refit_gap rejects a model whose profit lived in fitting on the whole file.
• --model-cutoff 2024-06. An edge that exists only before a language model's training cutoff is look-ahead through the model's memory.
• --events events.csv. News, earnings or LLM outputs are joined by publication time. The report shows what a join by event time would have earned.
• verify --returns manager.csv. Due diligence from returns alone: smoothing, missing small losses, implausible consistency, significance given the manager's number of funds.
• verify --legs legs.csv and --null-test pairs.py:pnl. Phantom profit in pairs and stat-arb strategies: a rolling hedge ratio booked as position times the change of the spread earns a profit that no trader could, even on independent random walks. --legs re-derives the P&L from the quantity held in each leg and rejects a reported return the legs never earned (and writes the corrected series); --null-test runs the strategy on random walks, where it must earn nothing. Try monte-neo verify --demo-phantom.
• Your institution's rules and history. --policy policy.toml applies your thresholds and records them in the certificate. --trials-from mlflow:./mlruns (Optuna, W&B, Ray Tune too) counts the runs you already logged. --ledger on a shared drive counts the whole team's attempts.
• Where you already work. %%verify in Jupyter, Polars frames as input, and monte-neo dossier --lite for a one-page plain-language summary for people who do not read Sharpe ratios.
• Two more lint rules for ML code. full_sample_transform (a scaler or PCA fitted on all rows) and target_encoding_full (a target mean over all rows mapped back to features).

Guides: ML and events · track records · phantom profit in pairs strategies · policy, trials, model cutoff · reference alpha.

Catch look-ahead that hides in milliseconds

[monte-neo verify --quotes: a fast strategy earns +191% on exchange time and loses 23% on arrival time; a slow honest strategy keeps its profit]

The problem. A quote has two times: when the exchange stamped it and when it reached you. Most backtests on intraday quotes bin them by the stamp, so a fast strategy trades on prices it could not have seen yet. No code lint finds this: the leak is in the clock, not in the code.

What Monte-Neo does (monte-neo verify --quotes quotes.csv --strategy my_strategy.py):

• Arrival look-ahead. The same strategy sees bars built from the arrival time and trades against the market. If the profit exists only with zero latency, it is a leak at millisecond scale.
• The delay where the profit vanishes. One number you can act on: "the profit disappears at 38 ms of extra delay".
• Latency Monte Carlo. Each quote's latency is redrawn from the latency you actually observed (inside its venue), 200 seeded draws: the distribution of the return and the probability of a loss. It shows when a profit was lucky latency.
• Order delay. --order-latency-ms delays every fill, so an edge that lives in instant execution is rejected.
• No server near the exchange? --latency-model lognormal:8,25 assumes the latency (median and p95 in ms) instead of reading it, and the certificate says plainly that it is assumed, not measured.
• Several feeds in one strategy. --symbol and --feeds let a strategy trade one instrument on the prices of another (lead-lag, futures against spot); each feed has its own latency. A leader that arrives after the follower moved is caught.
• Look-ahead probes too. The truncation and perturbation probes of verify_strategy run on the arrival bars, so a strategy that reads the next bar (shift(-1)) is rejected: that leak exists on every clock.
• Quote quality. Crossed quotes, negative latency, out-of-order arrivals and bursty latency: a congested path (VPN, Wi-Fi) holds messages and releases them in bursts, and the check says so instead of reporting your network as a result.
• A certificate. Signed, and reproducible with --recheck like every other certificate.

$ monte-neo verify --demo-quotes     # no files: a fast strategy and an honest one$ python -m monte_neo.data.quote_recorder --symbol BTCUSDT --seconds 600 --out quotes.csv   # your own latency

What sets it apart: it is an audit, not an environment. It takes a finished strategy from any source (any signal(df) function, or an agent's code) and your own quote recording, and says whether the profit survives the time the data arrived, with a certificate others can reproduce. The strategy does not have to be written for a particular engine.

Honest scope: experimental. The latency checks are context (warn at most) and were calibrated on synthetic data, so record real latency on the machine that will trade, or state an assumed latency and read the verdict as conditional on it. Fills are taker orders at the mid, with the data latency from the quotes (or assumed) plus a fixed order delay. Queue position and partial fills are not modelled, on purpose (they need the order book and a passive-order model; a rough formula would give false precision), so a market-making strategy cannot be validated here; the certificate lists these assumptions. Read the guide.

Search for an indicator, honestly

monte-neo discover --ohlcv data.csv --out found/

Tries thousands of causal formulas, counts the search, tests it against shuffled markets and a lockbox, and writes a certificate for the winner. On random walks it names nothing (0 of 60 searches in the calibration), although a plain search would report a Sharpe above 6. See the guide.

The rest of the workflow

• verify --ledger counts every variant you try; verify --suggest-fix rewrites leaks as a diff (guide).
• Signals must not repaint: a signal that changes after it was shown fails, and a signal that needs the bar's own prices is reported (guide).
• history, register, oracle, portfolio, doctor: certificate history with a regression exit code, pre-registered hypotheses, a Thresholdout hold-out for agents, many strategies at once, and known problems of data exports (guide).
• The honesty scorecard shows how often the verifier catches leaks and spares honest strategies (hypothesis zoo, with intervals and known gaps).

What it checks

Every rule is backed by the Trap Suite: 80 strategies that are known to lie and 35 honest controls. It runs on every build, so the verifier cannot silently stop catching a leak or start accusing honest code.

How it differs from other tools

Where to use it

Why Monte-Neo

• Independent. It checks code it did not write, with probes that do not trust the strategy's own numbers.
• Careful with accusations. 35 honest strategies (loops, windows, resampling, fits inside rolling windows, a real edge with a high hit rate) must never be flagged for look-ahead, on every build.
• Repairs without overfitting. repair fixes defects chosen by the number of defects, counts every attempt and lets a lockbox decide, and says so when the profit was the defect.
• Built for agents. An MCP server with 23 tools, a Claude Code plugin with a skill, a slash command and a reminder hook, plus rules for Codex, Gemini CLI, Cursor and Qwen Code. Every failed check returns a next_action the agent can act on.
• Institutional evidence. Alpha reports with a readiness level, validation dossiers mapped to model-risk frameworks, live monitoring, your own policy recorded in every certificate.
• Time-aware. It checks whether the profit survives the time the quotes really arrived, not only the time the exchange stamped them.
• Reproducible. The same data, code and n_trials always give the same certificate_id. Anyone can reproduce a certificate with --recheck.
• Signed. Ed25519 signatures show who issued a certificate and that nobody edited it.
• Honest about selection bias. Declare how many variants you tried, or let verify_grid count them for you. The Deflated Sharpe prices them in.
• Local and private. Your data and code never leave your machine. MIT licensed.

Quick start

pip install monte-neomonte-neo verify --demo      # no files needed: a strategy that peeks at tomorrow's close is rejectedmonte-neo init-ci            # write a GitHub Actions workflow that verifies your strategy on every pull request

Command line

monte-neo verify --ohlcv btc_1h.csv --strategy my_strategy.py --n-trials 12 --out verdict.json

my_strategy.py defines signal(df), which returns one position per bar: +1 long, 0 flat, -1 short, or a weight such as 0.5 (half the equity long). The position decided on bar t is filled at the open of bar t + 1.

def signal(df):    fast = df["close"].rolling(20).mean()    slow = df["close"].rolling(80).mean()    return (fast > slow).astype(int)

Python

from monte_neo.verify import verify_strategy
report = verify_strategy("btc_1h.csv", strategy="my_strategy.py", n_trials=12)print(report["verdict"], report["next_actions"])

Parameter search with honest trial counting

monte-neo verify --ohlcv btc_1h.csv --strategy sma.py --grid '{"fast": [10, 20], "slow": [80, 120]}'

Several symbols, and a report for people

# universe.csv: timestamp, symbol, open, high, low, close; signal(df) returns a weight per rowmonte-neo verify --ohlcv universe.csv --strategy xs_momentum.py --html report.html

The HTML report is a tear sheet in one file with no scripts: the reason for the verdict in one sentence, equity on a log scale against buy-and-hold, drawdown, rolling Sharpe, a monthly heat map, the timing test, sensitivity to costs, trade statistics, the evidence for a leak (bars and source lines), every check, results by period and market regime, and the hashes to reproduce the run. It prints to PDF from the browser. See the example reports.

Use it from your coding agent

Claude Code (plugin with the MCP server, the verify-strategy skill and /verify):

/plugin marketplace add NeoZorK/Monte-Neo/plugin install monte-neo@monte-neo

Any MCP client (Codex, Gemini CLI, Cursor, and others):

uvx monte-neo mcp

It is also listed in the official MCP Registry as io.github.NeoZorK/monte-neo. Setup for each client: Use from agents.

What was tested. The maintainer has run the server with Claude Code and with Qwen Code on a local model. Codex, Gemini CLI, Cursor and other MCP clients get the same server and rules, but nobody has reported a run yet. If you use one of them, a 10-minute test helps: issue #114.

Local agents (Qwen Code, or any terminal agent on a model served from your machine through Ollama, LM Studio or llama.cpp):

qwen mcp add monte-neo uvx --from "monte-neo[mcp]>=0.53.0" monte-neo-mcp

qwen-code is also a built-in agent of monte-neo bench run, so you can measure how honest a local model is. A small model often fails the bench tasks; that is a result of the model, not of the verifier. Setup, --base-url and troubleshooting: Local agents.

MCP tools (23): verify_strategy, verify_grid, verify_quotes, verify_portfolio, verify_returns, verify_legs, validate_alpha, repair_strategy, build_dossier, monitor_live, probe_lookahead, cost_stress, recheck_certificate, check_signature, render_report, suggest_fix, discover_indicator, register_hypothesis, compare_certificates, holdout_query, diagnose_data, verdict_schema, verifier_manifest.

GitHub Action

- uses: NeoZorK/[email protected]  with:    ohlcv: data/btc_1h.csv    strategy: strategies/momentum.py    n-trials: "12"    comment: "true"                                        # post the verdict on the pull request    signing-key: ${{ secrets.MONTE_NEO_SIGNING_KEY }}      # optional: sign the certificate    upload-certificate: "true"                             # optional: keep it as a workflow artifact

The job fails on REJECT. The verdict and every check appear in the step summary.

Certificates you can check

Each run produces a strategy-verdict/1 JSON certificate. It contains the verdict, every check, the metrics and the SHA-256 hashes of the data, signals and code.

monte-neo verify --recheck verdict.json --ohlcv btc_1h.csv --strategy my_strategy.py  # reproduce itpip install "monte-neo[sign]"monte-neo verify --keygen issuer                                  # issuer.key + issuer.pubmonte-neo verify --ohlcv btc_1h.csv --strategy my_strategy.py --sign issuer.key --out verdict.jsonmonte-neo verify --check-signature verdict.json --public-key issuer.pub

Show that a strategy passed:

[Verified by Monte-Neo]

[![Verified by Monte-Neo](https://img.shields.io/badge/verified%20by-Monte--Neo-2ea44f)](https://github.com/NeoZorK/Monte-Neo)

Link the badge to the verification page with your certificate and public key (/verify/?cert=<https URL>&key=ed25519:<key>). Readers then check the signature in their browser with one click; nothing is uploaded.

Install options

pip install monte-neo              # verifier, CLI and MCP serverpip install "monte-neo[sign]"      # + Ed25519 certificate signingpip install "monte-neo[parquet]"   # + read .parquet tablespip install "monte-neo[research]"  # + interactive research CLIpip install "monte-neo[plot]"      # + chartspip install "monte-neo[apple]"     # + Metal / MLX research engine (Apple Silicon)pip install "monte-neo[full]"      # everything

Python 3.11+ on macOS or Linux.

from monte_neo.backtest import ExecutionModel, export_sma_sweep, synthetic_ohlcv
ohlc = synthetic_ohlcv(100_000, seed=42)model = ExecutionModel(commission_bps=5.0, slippage_bps=5.0, warmup_bars=50)out = export_sma_sweep(ohlc["open"], ohlc["high"], ohlc["low"], ohlc["close"], combos=16, model=model, device="auto")print(out["device"], out["combos"])

Help the project

Monte-Neo is free and MIT-licensed, and it is built in the open. If it saved you from a bad backtest, three things help most:

• Star the repository on GitHub. Stars are how quants and agent builders find a verifier.
• Tell us about a case it caught or missed: share it in Discussions, or open an issue with a minimal strategy file. Missed leaks become new Trap Suite cases.
• Add the badge to your strategy repository (see the certificate badge section above), so readers know the result was verified.

Project status

Monte-Neo is in active development (beta). The verifier API and the strategy-verdict/1 schema are stable across minor releases. See the roadmap.

Not investment advice. Monte-Neo is a research and verification tool. Its reports, examples and reference strategies illustrate a method; they are not a recommendation, an offer or a call to buy, sell or trade any instrument, and not a promise of profit. Simulated and past results do not predict future returns. Trading can lose money, including more than the initial capital when leverage is used. Decisions and their consequences are yours; consult a licensed adviser if you need advice.

Contributing

Found a way a backtest fooled you or your agent? Submit it as a trap. Did the verifier accuse an honest strategy? Report a false accusation. Bug reports and pull requests are welcome; see the contributing guide. Report security issues privately: SECURITY.md.

Citation

The method is described in the technical note.

If Monte-Neo helps your research, please cite it. GitHub shows the citation under Cite this repository (CITATION.cff).

License

MIT