In one sentence
Bootstrapping draws thousands of new "alternative histories" from your actual bets, with replacement, to measure how uncertain your ROI or model score really is.
How it works
You have 400 bets and a profit. The question is how different that profit could have looked with slightly different luck. Bootstrapping answers it by treating your 400 bets as a bag and drawing 400 bets from it at random, putting each one back after drawing.
Some bets get drawn twice, some not at all. Work out the ROI for that resampled record, then repeat 10,000 or more times. The spread of those ROIs shows the range your true performance could plausibly be in.
It needs no formula for the distribution of returns, which is handy because betting returns are lumpy and skewed, especially at bigger odds.
The maths
- n is the number of bets; s is the level stake.
- R-star with subscripts b and i is the profit or loss of the i-th bet drawn in resample b.
- ROI-star with subscript b is the return on investment in resample b.
- B is the number of resamples; the 2.5th and 97.5th percentiles of the B values give a 95% interval.
In words: rebuild your record many times from its own bets, and look at how much the ROI moves around.
Worked betting example
A football tipster has 400 level-stake £10 bets on Match Odds away wins, all at 3.0. There were 140 winners.
- Profit = 140 × £20 − 260 × £10 = £2,800 − £2,600 = £200.
- ROI = £200 ÷ £4,000 staked = +5.0%.
- Bootstrap with 20,000 resamples: the middle 95% of ROIs runs from about −9.3% to +19.3%.
- About 25% of the resamples show zero or negative profit.
The standard deviation of a single bet's return here is about £14.33, so the standard error of the ROI is about 7.2 percentage points. To get a 5% ROI two standard errors clear of zero at these odds, you would need roughly 3,300 bets.
In plain terms: 400 bets at 3.0 cannot tell a +5% tipster from a break-even or losing one. With 2% commission on each winner the profit is 140 × £19.60 − £2,600 = £144, an ROI of +3.6%, so the picture is weaker still.
Where it's good
- Putting a confidence interval on your own ROI or a tipster's record before paying for it.
- Comparing two models' log loss or Brier score by resampling matches, to see if the difference is real.
- Estimating the likely range of drawdowns from a strategy's historical bets.
- Getting uncertainty for model parameters that lack a textbook formula.
Limitations and pitfalls
- Bootstrapping cannot fix a biased sample. If the 400 bets were selected after the fact, or the backtest leaks future data, the interval is precise about the wrong thing.
- It assumes bets are independent. Bets on the same match, same day or same systematic angle are correlated; resample by day or by block instead, or the interval will be too narrow.
- It says nothing about the future if the edge has decayed. Markets adapt, and past resamples cannot capture that.
- Rare big-priced winners dominate results; with few of them the bootstrap is unstable too.
- The share of negative resamples is not a p-value in the strict sense. For a formal test of "no edge", use a permutation test or hypothesis test.
- If you tested many strategies and are bootstrapping the best one, the interval is too optimistic. Account for the search.
How to build it
- Python: numpy's random choice, or scipy.stats.bootstrap; R: the boot package.
- Data: one row per bet with stake, odds, result and commission paid, plus date for block resampling.
- Tip: include commission in each bet's return before resampling, and use at least 10,000 resamples.
Related methods
- P-values and confidence intervals give the formula-based version.
- Permutation tests test whether an edge is zero.
- Monte Carlo simulation simulates from a model rather than from data.
- Hypothesis testing frames the "is it luck?" question formally.
- Drawdown analysis often uses bootstrapped bet sequences.