In one sentence
Natural language processing (NLP) converts written text into numbers, such as a sentiment score or a flag that a key player is out, so it can feed a model.
How it works
Much of what moves prices starts as words: a manager hints a striker is doubtful, another says he will rotate for the cup, a journalist reports a late fitness test. NLP reads that text at scale.
Simple methods count positive and negative words from a list. Modern methods use pre-trained transformers that understand context, so "not bad at all" is read as positive. They can also pull out specific facts, such as which players are named as injured.
The output is usually a feature that adjusts a baseline model's probability. The big question is timing: by the time most text is public, the exchange has already reacted.
The maths
- S is a sentiment score between −1 (all negative) and +1 (all positive).
- n-plus and n-minus are counts of positive and negative words or posts.
- logit(p) is the log-odds, the log of p divided by 1 minus p.
- β (beta) is how much one unit of sentiment moves the log-odds, estimated from past data.
In words: score the text, then shift the model's log-odds by an amount learned from how sentiment related to results in the past.
Worked betting example
A baseline model gives an away side a 45% chance. The morning's team news and reporter posts contain 12 positive mentions (key midfielder fit, strong training reports) and 4 negative.
- S = (12 − 4) ÷ (12 + 4) = 0.50.
- Base log-odds = log(0.45 ÷ 0.55) ≈ −0.201.
- With β = 0.3 estimated from past seasons: new log-odds = −0.201 + 0.3 × 0.50 ≈ −0.051.
- New probability = 1 ÷ (1 + e to the power 0.051) ≈ 48.7%.
The away side is 2.30 in Match Odds on Betfair, implying about 43.5%.
- A £10 back wins £13. EV = 0.487 × £13 − 0.513 × £10 ≈ +£1.21, or about +£1.08 after 2% commission.
The catch: if that price was taken after the news broke, it has probably already moved. Backtests must use the price available when your system could actually have read the text.
Where it's good
- Extracting structured facts from team news: confirmed absentees, returning players, goalkeeper changes.
- Monitoring many sources faster than a person can, especially in lower-profile markets where prices react slowly.
- Parsing press-conference quotes into features such as "rotation expected" or "back from injury".
- Adding context to anomaly alerts: explaining why a market just moved.
Limitations and pitfalls
- Speed matters more than cleverness. Professional operations read team news in seconds; a slower sentiment model trades on stale information.
- Timestamp leakage is the classic mistake: joining tweets to matches by date, not by exact time, lets post-kick-off text leak into pre-match predictions.
- Social media sentiment is mostly fans being fans. Crowd optimism rarely carries information the market lacks, and may reflect bias you want to fade, not follow.
- Lexicon methods miss sarcasm, negation and sport-specific language ("he's a doubt" versus "no doubt").
- Large language models can invent facts. Always verify extracted team news against an official source before staking.
- Estimating β needs a lot of matched text and results, and the relationship drifts as sources change.
How to build it
- Python: Hugging Face transformers for pre-trained sentiment and entity models; VADER or spaCy for fast lexicon and rule-based work.
- Data: time-stamped text (club sites, verified reporters, press-conference transcripts) and the exchange price at each timestamp.
- Tip: measure how many seconds after publication the Betfair price moves; if it moves before your system could act, the feature has no trading value.
Related methods
- Transformers power modern text models.
- Logistic regression is the usual way to add a sentiment adjustment.
- Bayesian updating frames news as evidence that shifts a prior.
- Market efficiency explains why public news is priced in fast.
- Informed trader models describe who profits from early information.