Backtest Overfitting Audit

io.github.tylerscomic-labv1.0.0更新於 Oct 2, 2026

Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, and purged CV splits.

已驗證Streamable HTTP可網頁執行FinanceData & Analytics

概覽

AI 產生的概覽

以 CSCV 機率、通縮夏普比率、最短回測長度與淨化交叉驗證切分，稽核交易回測是否過度擬合。

功能
此伺服器對策略回測執行統計檢定。其工具以組合對稱交叉驗證（CSCV）估計回測過度擬合機率，計算依試驗次數與報酬非常態性調整的通縮夏普比率，給出夏普比率具統計意義所需的最短回測長度，並為時間序列資料產生淨化並設禁區的交叉驗證切分。它針對的是在同一段歷史上測試大量參數變體、僅因偶然得到漂亮結果的失效模式。
適用情境
適合在評估量化交易策略、需要判斷表面優勢能否在樣本外成立時使用，或已針對同一段歷史跑過大量參數組合時。也適合為時間序列回測建立無前視洩漏的交叉驗證切分。
執行需求
遠端 streamable HTTP 端點；未宣告驗證、環境變數或標頭。README 另描述以 npm install 與 node server.js 自行架設的方式，需要 Node.js。代管方式據說明有免費方案與付費 Pro 方案。
安裝前請注意
README 提到免費方案與用於提高額度的付費 Pro 方案，因此代管使用可能涉及付費或用量限制。提交分析的回測資料會傳送到遠端端點。清單未要求任何憑證。

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 Backtest Overfitting Audit，將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Web executable，透過 Streamable HTTP。 遠端服務在工作區中設定後即可從網頁執行環境執行。

其他 MCP 客戶端

把它新增到你客戶端的 mcpServers 設定中。

{
  "mcpServers": {
    "overfitting-audit-mcp": {
      "type": "http",
      "url": "https://overfitting-audit-mcp.mcpize.run/mcp"
    }
  }
}

README

overfitting-audit-mcp

[License: MIT] [Live on MCPize]

An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.

The problem this solves

Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.

Tools

probability_of_backtest_overfitting

Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.

deflated_sharpe_ratio

Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.

minimum_backtest_length

The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.

purged_cv_split

Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.

Use it

Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.

Self-host:

bash
npm installnode server.js

Part of the AlgoForge suite

Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.

License

MIT

來源：README.md，提交 c0087b1

工具

0
工具後設資料尚未被收錄。

版本歷史

1
  1. v1.0.0最新Oct 2, 2026