Backtest Overfitting Audit

io.github.tylerscomic-labv1.0.0更新于 Oct 2, 2026

Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, and purged CV splits.

已验证Streamable HTTP可网页运行FinanceData & Analytics

概览

AI 生成的概览

使用 CSCV 概率、通缩夏普比率、最短回测长度和净化交叉验证切分来审计交易回测是否过拟合。

功能
该服务器对策略回测实施统计检验。其工具通过组合对称交叉验证（CSCV）估计回测过拟合概率，计算按试验次数和收益非正态性调整的通缩夏普比率，给出夏普比率具有统计意义所需的最短回测长度，并为时间序列数据生成净化并设置禁区的交叉验证切分。它针对的是在同一段历史上测试大量参数变体、仅凭偶然得到好看结果的失效模式。
适用场景
适用于评估量化交易策略、需要判断表面优势能否在样本外成立时，或已针对同一段历史运行了大量参数组合时。也适合为时间序列回测构建无前视泄漏的交叉验证切分。
运行要求
远程 streamable HTTP 端点；未声明认证、环境变量或请求头。README 还描述了通过 npm install 和 node server.js 自托管的方式，需要 Node.js。托管方式据说明有免费档和付费 Pro 档。
安装前请注意
README 提到免费档和用于更高限额的付费 Pro 档，因此托管使用可能涉及付费或用量限制。提交分析的回测数据会发送到远程端点。清单未要求任何凭据。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Backtest Overfitting Audit，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable，通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "overfitting-audit-mcp": {
      "type": "http",
      "url": "https://overfitting-audit-mcp.mcpize.run/mcp"
    }
  }
}

README

overfitting-audit-mcp

[License: MIT] [Live on MCPize]

An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.

The problem this solves

Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.

Tools

probability_of_backtest_overfitting

Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.

deflated_sharpe_ratio

Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.

minimum_backtest_length

The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.

purged_cv_split

Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.

Use it

Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.

Self-host:

bash
npm installnode server.js

Part of the AlgoForge suite

Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.

License

MIT

来源：README.md，提交 c0087b1

工具

0
工具元数据尚未被收录。

版本历史

1
  1. v1.0.0最新Oct 2, 2026