Walk-Forward Validation

In one sentence

Walk-forward validation trains your model only on data from before each test period, bets through that period, then moves the window forward and repeats, so no result is ever predicted using information from its own future.

How it works

Ordinary cross-validation shuffles data into random folds. That is fine for photos of cats but wrong for sport, because it lets a model learn from next season's matches while "predicting" this season's. Team strength, rules and market efficiency all change over time, so leaking the future flatters results.

Walk-forward keeps time in order. Train on 2018-2020, test on 2021. Then train on 2018-2021 (an expanding window) or 2019-2021 (a rolling window), test on 2022, and so on. Each test season is a fair rehearsal of what you would have done live.

You then join all the test periods together and judge the model on those results only. If your in-sample backtest looked brilliant but the walk-forward results are thin, the gap is a measure of how much you overfitted.

The maths

There is no single formula; the method is a procedure. The pooled result across folds is:

• ROI WF: the walk-forward return on investment.
• K: the number of test folds, for example seasons.
• profit and staked: the net profit and total stakes in test fold k.

In plain English: add up all profit from the out-of-sample periods and divide by all money staked in them.

Worked betting example

A football draw model, £10 level stakes, Betfair prices, 2% commission already deducted. The figures are illustrative. On all data at once (in sample), the backtest showed an eye-catching ROI. Walk-forward with an expanding window gives:

Step by step:

1. Total bets: 410 + 395 + 430 + 402 = 1,637, so £16,370 staked.
2. Total profit: 127.10 − 71.10 + 51.60 + 96.48 = £204.08.
3. Pooled ROI: 204.08 ÷ 16,370 ≈ 1.25%.

A 1.25% ROI over 1,637 bets at draw prices (typically well above 3.0) is well within what luck alone produces. The honest conclusion is "possibly a small edge, not proven", which is very different from what the in-sample figure suggested.

Where it's good

• Any model where the world changes over time: football team ratings, league goal rates, in-play trading rules.
• Choosing settings such as how quickly ratings decay, without peeking at the test period.
• Estimating how a model's edge has changed as markets have become more efficient.
• Producing a realistic record to take into significance testing and staking decisions.

Limitations and pitfalls

• If you run walk-forward many times, tweaking features after each run, the test periods quietly become training data. Keep a final hold-out period you only look at once.
• Short folds make results noisy. A single season of a few hundred bets rarely proves anything on its own.
• It does not fix other backtest biases: using closing prices you could not have taken, ignoring commission, or survivorship in your data.
• Expanding windows can underweight recent changes; rolling windows can throw away useful history. Try both.
• Retraining is not free in the live world; match your walk-forward retraining schedule to what you would actually do.
• Features must be calculated only from information available at bet time, including within the test fold.

How to build it

• Python: sklearn.model_selection.TimeSeriesSplit, or a simple loop over seasons with pandas date filters.
• Data: timestamped matches, the odds you could actually have taken at the time, and commission rates.
• Tip: record the date each feature was known, not just the match date, so late-arriving data such as final xG totals cannot leak in.

Related methods

• Backtest bias checks - the other traps walk-forward does not catch.
• Hyperparameter optimisation - tuning must sit inside each training window.
• Model vs market - judge each fold against the price, not just profit.
• P-values and confidence intervals - is the pooled ROI more than noise?
• Regularisation - reduces the overfitting that walk-forward exposes.