Backtesting Frameworks

by wshobson46891e7e60daNo licenseListed Oct 8, 2026Updated Oct 8, 2026

Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.

Instructions onlyBusiness & Finance

AI-generated overview

Guides building robust trading-strategy backtesting systems that avoid bias and model costs realistically.

What it does: This skill provides guidance for constructing production-grade backtesting systems for trading strategies. It explains common backtesting biases such as look-ahead, survivorship, overfitting, selection and transaction-cost bias, and gives mitigations. It outlines proper train/validation/test structure, walk-forward analysis, and best practices, with detailed worked examples and patterns in a separate reference file.
When to use it: Use it when developing trading algorithms, building backtesting infrastructure, validating strategy performance, or comparing strategy alternatives. It is also relevant when implementing walk-forward analysis or trying to avoid common backtesting biases.
Requirements: No scripts are included; it is instructions only. It relies on a reference file (references/details.md) for detailed worked examples and patterns.

Backtesting Frameworks

Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.

When to Use This Skill

Developing trading strategy backtests
Building backtesting infrastructure
Validating strategy performance
Avoiding common backtesting biases
Implementing walk-forward analysis
Comparing strategy alternatives

Core Concepts

1. Backtesting Biases

Bias	Description	Mitigation
Look-ahead	Using future information	Point-in-time data
Survivorship	Only testing on survivors	Use delisted securities
Overfitting	Curve-fitting to history	Out-of-sample testing
Selection	Cherry-picking strategies	Pre-registration
Transaction	Ignoring trading costs	Realistic cost models

2. Proper Backtest Structure

Historical Data      │      ▼┌─────────────────────────────────────────┐│              Training Set               ││  (Strategy Development & Optimization)  │└─────────────────────────────────────────┘      │      ▼┌─────────────────────────────────────────┐│             Validation Set              ││  (Parameter Selection, No Peeking)      │└─────────────────────────────────────────┘      │      ▼┌─────────────────────────────────────────┐│               Test Set                  ││  (Final Performance Evaluation)         │└─────────────────────────────────────────┘

Historical Data      │      ▼┌─────────────────────────────────────────┐│              Training Set               ││  (Strategy Development & Optimization)  │└─────────────────────────────────────────┘      │      ▼┌─────────────────────────────────────────┐│             Validation Set              ││  (Parameter Selection, No Peeking)      │└─────────────────────────────────────────┘      │      ▼┌─────────────────────────────────────────┐│               Test Set                  ││  (Final Performance Evaluation)         │└─────────────────────────────────────────┘

3. Walk-Forward Analysis

Window 1: [Train──────][Test]Window 2:     [Train──────][Test]Window 3:         [Train──────][Test]Window 4:             [Train──────][Test]                                     ─────▶ Time

Window 1: [Train──────][Test]Window 2:     [Train──────][Test]Window 3:         [Train──────][Test]Window 4:             [Train──────][Test]                                     ─────▶ Time

Detailed worked examples and patterns

Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices

Do's

Use point-in-time data - Avoid look-ahead bias
Include transaction costs - Realistic estimates
Test out-of-sample - Always reserve data
Use walk-forward - Not just train/test
Monte Carlo analysis - Understand uncertainty

Don'ts

Don't overfit - Limit parameters
Don't ignore survivorship - Include delisted
Don't use adjusted data carelessly - Understand adjustments
Don't optimize on full history - Reserve test set
Don't ignore capacity - Market impact matters

Source and attribution

Source:wshobson/agentsinplugins/quantitative-trading/skills/backtesting-frameworksat commit46891e7

License: No license

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

Backtesting Frameworks

Backtesting Frameworks

When to Use This Skill

Core Concepts

1. Backtesting Biases

2. Proper Backtest Structure

3. Walk-Forward Analysis

Detailed worked examples and patterns

Best Practices

Do's

Don'ts

Source and attribution

More from wshobson/agents

Uv Package Manager

Web Component Design

Python Type Safety

Visual Design Foundations

Responsive Design

Python Resource Management

More in Business & Finance

Gke Cost Optimization

Cost Optimization

Cost Governance

Cost Estimation

Cost Analysis

Google Ads Api Account Diagnostics