Why good models still lose money

Statistical modelling is the most powerful tool a bettor or trader has. It's also where most serious bettors lose their money, and understanding the limitations of betting models is the first step to not being one of them.

That isn't because the maths is wrong. A model is a simplified picture of the world, and the market is full of people with their own pictures, some of them better than yours. This article covers where models break, why backtests lie, and what you need to prove before you stake a penny.

The short version is that a model doesn't create an edge. It measures one. If the edge isn't there, no amount of maths will put it there.

Pitfall 1: You're not betting against the sport. You're betting against the price.

Most people build a model to answer "who will win?" That's the wrong question. The right question is: "Is the market's price wrong, and by enough to beat commission?"

A model can be accurate and still lose. Say your model gives Liverpool a 62% chance and it's right. If the market has them at 1.55 (64.5%), you have no bet. The market was more accurate than you.

Betfair's closing prices in the major football markets are some of the most accurate forecasts in the world. Thousands of people, many with better data and faster feeds than you, have already shaped that price. Your model has to beat that crowd, not just be sensible.

What to do: always compare your model with the market price, not with results. The best single test of a model is closing line value (CLV): whether you consistently get better odds than the final price. → Lesson 2.4: Closing line value

Pitfall 2: Not enough data, and too much noise

A Premier League team plays 38 league games a season. That feels like data, but statistically it's a thimbleful. Football results are driven heavily by luck: one deflection, one penalty, one red card.

Small samples plus lots of noise means your model can easily "learn" patterns that are just luck.

What to do: use models that pull small samples towards sensible averages (shrinkage and hierarchical models). Prefer underlying measures like expected goals over raw results, because they carry more signal per game. → Lesson 5.2: Shrinkage

Pitfall 3: The past isn't the future

Every model assumes tomorrow looks roughly like yesterday. In football, that assumption keeps breaking:

• Managers change and tactics change with them.
• Squads turn over every summer.
• Rules change: VAR, five substitutes, the added-time crackdown.
• Markets change as well. Once an edge becomes known, the price adjusts and the edge disappears.

A model trained on 2019–2022 data may be describing a game that no longer exists.

What to do: give recent data more weight, retrain regularly, and watch your live performance for sudden changes. → Lesson 7.3: Spotting when your edge has gone

Pitfall 4: The model's assumptions are simplifications

Every model rests on assumptions. The classic Poisson football model assumes goals arrive independently at a steady rate. In reality a team 1-0 up at 70 minutes sits deeper, and the trailing team throws bodies forward.

Most statistics also assume the bell curve. Real betting returns have fat tails: red cards, injuries, late collapses and market shocks happen far more often than the bell curve predicts.

What to do: know what your model assumes, and test whether it holds. Check that predicted 0-0s match actual 0-0s, for example. Stress-test your staking against the rare disasters, not just the average case. → Lesson 1.4: Fat tails

Pitfall 5: Overfitting, the biggest killer of all

This is how most "profitable systems" are born and then die.

Test enough ideas on historical data and some will look brilliant by pure chance. We ran a simulation: 100 betting systems with zero edge, each tested on 200 bets at evens. The best-looking system showed a median ROI of +17%, and in every run at least one worthless system showed +10% or better.

The more ideas you test, the more fake winners you find. Complex machine learning models make this worse, because they're extremely good at memorising noise.

Warning signs of an overfitted model:

• Amazing backtest, poor live results.
• Very specific rules, e.g. "Away teams, Tuesdays, odds 2.2–2.6, after a draw."
• The edge disappears when you change the dates slightly.
• You can't explain why the edge should exist.

What to do:

• Split your data by time. Build on the past, test on a period the model has never seen.
• Correct for the number of ideas you tested.
• Demand a reason for the edge: who is on the wrong side of this bet, and why?

→ Lesson 7.4: Overfitting and multiple testing

Pitfall 6: Backtests cheat unless you stop them

A backtest is only as honest as what you let it know. The two ways it cheats are the price and the information.

• Commission. Test after Betfair's 2%. A +5% ROI at average odds of 3.0 is about +3.6% once commission comes off the winners.
• The price. Use the price that was actually there when you would have placed the bet, not the best price the market showed all day, and not the closing price.
• Data leaks. Using anything the model couldn't have known at the time, such as the final team sheet, the closing price or a later result, makes any backtest look incredible.
• Speed. In-play edges usually go to whoever is fastest. Know whether your strategy needs to be quick, and test it at the speed you can actually act.

What to do: backtest at the price available at bet time, with commission, using only information known at that moment. → Lesson 7.5: Building an honest backtest

Pitfall 7: Variance will test your nerve

Even with a real edge, results are brutally noisy. Take a genuine +5% edge after commission, betting at odds of 3.0:

The longest losing run you should expect over 1,000 bets is about 14 in a row. That's a normal run for a winning strategy.

This creates two opposite mistakes:

1. Giving up on a good model during a normal losing run.
2. Trusting a bad model because it started with a lucky run.

The only protection is knowing, before you start, what normal bad luck looks like, and sizing your stakes so you survive it.

What to do: use fractional Kelly or fixed small stakes, know your expected drawdown in advance, and judge your model on hundreds or thousands of bets, never dozens. → Lesson 6.4: Drawdowns and losing runs

What statistics can't tell you

Statistics can tell you that a pattern exists. They can't tell you why, or whether it will continue. Correlation isn't causation, and a model can't see a manager's team talk or a side that has one eye on a cup final.

So where does a real edge come from?

Real edges almost always come from at least one of these:

1. Better or faster information than the market.
2. Less efficient markets: lower leagues, early prices before the big money arrives.
3. Better execution: timing, and getting matched at good prices.
4. Repeatable human bias: favourites and longshots mispriced, big-name teams overbet, overreaction to recent results.

The model's job is to find where one of these exists and put a number on it. Then statistics proves whether it's real.

The honest checklist before you stake real money

• My model beats the market price, not just the results.
• I tested on data the model never saw during building.
• My backtest uses the price available at the time, after commission.
• I corrected for how many ideas I tried.
• I can explain why the edge exists.
• I'm beating the closing line consistently.
• I know what a normal losing run looks like for my strategy.
• My stakes are sized so I survive that losing run.

If you can't tick every box, you don't have an edge yet. You have a hypothesis. Statometrics Academy teaches you to test it properly, one lesson at a time.