Canli Backtest

canli-backtest-mcp

Backtests an AI agent can trust. An MCP server that builds point-in-time factors from SEC filings, runs cross-sectional backtests of any signal on your own prices with no lookahead (enforced in code), charges costs on the actual trades, and records every variant in a trial ledger, so the best of a search is judged against the whole search instead of alone.

Local stdio only: your prices, signals and ledgers never leave your machine.

canli-backtest-mcp on canlicapital.com · All Canli Capital MCP servers. Free, MIT-licensed. For every server behind one install, use canli-mcp.

Why

Three mistakes make most agent-run backtests look better than they are:

1. Lookahead. Using a number before it was public: an annual figure in January that was filed in February, or a value that was later restated. pit_factor only uses what had been filed with the SEC by each date, as it stood then; backtest_signal only trades on signal rows dated at least lag_days trading days earlier, and stops if anything reaches past that.
2. Free trading. Ignoring turnover. Positions drift with prices between rebalances, and costs are charged on the trades from those drifted positions to the new targets.
3. Counting only the winner. An agent tries 200 variants and reports the best. With ledger, every run is recorded (in canli-validation-mcp's hash-chained ledger), and each reply gives the best run's deflated Sharpe after counting all of them, and the Sharpe that luck alone reaches.

Tools

Breadth lab, sweeps and labelled data

Size, crises and cross-validation

A backtest that assumes any size fills at the close flatters itself, so:

• Size. Give backtest_signal a capital_usd and a dollar_volume_file. An order fills only up to max_participation of recent daily volume (10% unless you say otherwise) and pays square-root impact, with volume and volatility read from the days before each trade. The reply shows the Sharpe at $100k to $1B, and no Sharpe for a size the market could not take. Order-book depth is not modelled: it needs paid data.
• Crises. Every reply holds the result against named crises, from the dot-com bust to the 2023 bank run. A crisis its dates do not fully cover is listed as untested, never as passed. crisis_replay does this for any returns file.
• Cross-validation. cpcv takes every variant's returns and, for each choice of held-out blocks, chooses the best on the rest, with purge and embargo at the edges, then scores it on what was held out. You get a spread of out-of-sample results, not one path. Splits can run as shards on other machines and merge back.

A record, a league and a paper market

• Proof ledger. proof_filings writes what companies filed into an append-only file, each line chained to the last and each batch signed, with a line for every figure a later filing changed. proof_file adds any file's hash, proof_verify checks the whole file and says which line first fails. It shows lines were not altered, not that a figure is right, and no record means only no record. Publish the head somewhere else, or the key holder could rewrite it all. The published ledger does that with a public attestation each month.
• League. league_new, league_commit and league_score run a tournament scored after the fact: an entrant signs and commits a hash before a round closes and shows the file afterwards, and anyone with the files can rerun the table. Each record is set against the best luck gives across that many entrants. The top entrants, chosen on earlier rounds only, form a paper book. No prizes, no stakes, nobody's money.
• Arena. arena_new, arena_orders and arena_step are a paper market for agents to trade in, cleared in a call auction, reproducible from a seed. Made-up assets and made-up money: it cannot reach anything real.
• Agent exchange. exchange_new, exchange_join, exchange_view, exchange_orders, exchange_step and exchange_report are a closed paper market where different AI agents and scripted traders meet in limit order books, with margin rules and forced selling, so a shock can become a cascade. exchange_run runs a whole market of scripted traders in one call. Time moves in steps, each agent sees only its own account, and a hash of the tape lets anyone check a run. Made-up assets and money. The rules are in docs/AGENT_EXCHANGE.md.
• Consensus. agent_consensus combines agents' probabilities and counts how many independent views they really hold, from how their errors move together: five models that share their mistakes are not five votes.
• Diversifiers. find_diversifiers ranks candidates by the independent bets they add and how they behave on the book's worst days. Return is shown last and never used to rank.

Quick start

npx -y canli-backtest-mcp

Claude Code:

claude mcp add canli-backtest -- npx -y canli-backtest-mcp

Then ask: "Build roa for these 200 tickers from 2015 to 2024, backtest it monthly on prices.csv with 10 bps costs, try quantiles 0.1, 0.2 and 0.3 in the ledger 'roa-search', and tell me whether the best one survives the search."

Files

Both files are CSV with a date column (YYYY-MM-DD) and one column per ticker:

• prices: closing prices, one row per trading day. Bring your own; adjust for splits and dividends if you want total returns. The server does not supply prices (vendor licences forbid passing them on).
• signal: one row per date the values became known. pit_factor writes this; your own signal works too, as long as each row is dated by when it was available.

Empty cells are missing values, never zero.

What a result does not establish

• Prices are yours: survivorship, splits, dividends and data errors in them pass straight into the result.
• A backtest measures one path of history under these costs; it does not establish that the signal keeps working.
• Costs are a flat rate per unit traded; market impact, short-borrow fees and financing are not charged unless included in cost_bps.
• SEC values are what companies reported in XBRL, as filed; late filers, non-XBRL filers and tag changes can be missing.

Configuration

Part of the Canli Capital MCP family

canli-validation-mcp (validators and the trial ledger), canli-fundamentals-mcp (SEC fundamentals point in time), canli-research-mcp (the open research record). Source: github.com/arhancanli/canlicapital. MIT licensed. Built by Arhan Canli.