Backtest Bias Checks explained for betting

In one sentence

Backtest bias checks are the routine tests you run to make sure a historical simulation of a strategy uses only information, prices and costs that would have been available in real life.

How it works

A backtest replays history and asks "what if I had bet this way?". It is the cheapest research tool you have and the easiest to fool yourself with. Almost every bias pushes results in the same direction: too optimistic.

The main culprits are look-ahead bias (using information from after the bet, such as final team news or the closing price), survivorship bias (data that quietly drops abandoned matches or relegated teams), data snooping (trying hundreds of variations and keeping the best), and cost blindness (forgetting commission).

Each has a simple check. The discipline is running all of them every time, especially when the result looks exciting.

The maths

There is no single formula; this is a checklist. The key cost adjustment, for a single back bet in a market, is:

\text{ROI}_{\text{real}} = p \,(O_{\text{taken}} - 1)(1 - c) - (1 - p)

ROI real: return per £1 staked after realistic costs.
p: the true win rate of your selections.
O taken: the average odds you could actually have matched, not the best or closing price.
c: commission on net winnings in that market, 0.02 at a 2% rate.

In plain English: price your backtest at odds you could really have got, and take commission off the winnings.

Worked betting example

A football backtest of away-win backs on Match Odds shows 500 bets with a 27% strike rate at average Betfair closing odds of 4.0. Stakes are £10, and the figures are illustrative.

Headline, at closing prices with no commission: 0.27 × 4.0 − 1 = +8.0% ROI, or +£400.
Add 2% commission: winners net 3 × 0.98 = 2.94 stakes. ROI = 0.27 × 2.94 − 0.73 = +6.38%, or +£319.
Use prices you could have taken: your rule fires an hour before kick-off, not at the close. Checking the price history, the average price at that moment was 3.8. ROI = 0.27 × 2.8 × 0.98 − 0.73 = +1.09%, or +£54.40.

The same selections go from +£400 to +£54 purely by using the price that existed when the bet was placed, and +1% over 500 bets is well within what luck produces. And if your filter used any information only known at kick-off, such as confirmed line-ups, the 27% strike rate is also in doubt.

Where it's good

Every backtest, before any real money: pre-match, in-play and trading strategies alike.
Reviewing a system you bought or saw online, whose results were almost certainly not bias-checked.
In-play and trading bots, where the exact moment each price was available matters hugely.
Comparing a backtest with the first few months of live results to find which bias you missed.

Limitations and pitfalls

Look-ahead bias: every feature must carry a timestamp of when it was known. Final xG, confirmed line-ups and starting prices are classic leaks.
Price timing: use the price at the moment your rule fires, not the closing price or the best price seen during the day.
Commission: take 2% off net winnings in every market, as in the example.
Survivorship and missing data: void bets, abandoned matches and teams that left the league must stay in the data with their real outcome.
Data snooping: each variation you try increases the chance of a lucky winner. Count your attempts and correct for them.
Regime change: results from years when markets were less efficient may not carry forward.

How to build it

Python: pandas for timestamped joins (merge_asof is ideal for "last known value before bet time"); Betfair historical data files for time-stamped prices.
Data: time-stamped prices, your commission rate, and results including voids.
Tip: compare the price you backtested at with the closing price. If the edge only exists at prices that were not there when your rule fired, treat it as unproven.

Walk-forward validation - keeps model training out of the test period.
Multiple testing corrections - the fix for data snooping.
Commission-adjusted EV - getting the cost side right.
Closing line value - a live check that your backtest edge is real.
Market impact - how your own stakes change the prices you get.