Walk-forward and purged cross-validation

Standard $k$-fold cross-validation shuffles data into random folds. With financial time series that is a mistake: neighbouring observations are correlated, labels often span several bars, and the future must never inform the past.

Train, validate, test

The minimum structure for any research project:

• Training set: fit the model or choose the parameters.
• Validation set: compare candidate models and tune.
• Test set: touched once, at the end, to estimate real performance.

If you look at the test set and then go back and change the model, it has become a validation set, and you no longer have an unbiased estimate.

Walk-forward analysis

Simulate how you would actually run the strategy through time:

1. Fit on a window of history.
2. Trade the next out-of-sample window with the fitted parameters.
3. Roll forward and repeat.
4. Stitch the out-of-sample windows together into one equity curve.

Windows can be anchored (training always starts at the beginning and grows) or rolling (fixed length, slides forward). Rolling adapts faster to regime change; anchored uses more data. The stitched out-of-sample curve is the closest thing a backtest gives you to a live track record.

The leakage problem

Many labels look forward in time — “did price rise over the next 10 bars?” An observation at time $t$ and one at $t+3$ then share seven bars of future information. If one is in training and the other in testing, the model has partially seen the test answer.

Purging and embargo

López de Prado’s fix for overlapping labels:

• Purging: remove from the training set any observation whose label period overlaps with any test observation’s label period.
• Embargo: additionally remove a small buffer of observations immediately after each test fold, because features like moving averages carry information forward.

Combinatorial purged cross-validation (CPCV) goes further: split the data into $N$ groups, test on every combination of $k$ groups, and recombine the out-of-sample pieces into many alternative backtest paths. Instead of one equity curve you get a distribution of them — much harder to fool yourself with.