Our model vs the bookmakers: 18,807 matches tested
By ProbaPredict Data Desk · Last updated · Data: 18,807 matches, 2022-23 to 2025-26
ProbaPredict's analysis of 18,807 matches found that margin-free bookmaker closing odds were 2.8% more accurate than our model on match results (Brier score 0.592 vs 0.609) and 2.7% more accurate than it on Over/Under 2.5 goals. The gap was smallest in the Championship and widest in the Premier League.
Key findings
- 1X2 Brier score: closing odds 0.592, our model 0.609 (lower is better), so closing odds were 2.8% more accurate than the model. Log loss: 0.991 vs 1.016 (2.5% gap).
- Over/Under 2.5 goals (17,344 matches with totals odds): closing odds 0.239, our model 0.246, a 2.7% gap.
- Among the leagues we publish, closest: Championship (+2.0%). Widest gap: Premier League (+3.5%).
- The model and the market disagreed by 5 points or more on at least one outcome in 56.0% of matches (10,533). In those matches the market gave the actual result a higher probability 62.4% of the time, the model 37.6%.
- When our model rated an outcome 5+ points higher than the market (average 39.5% vs 30.4%), it happened 28.2% of the time, closer to the market's figure.
What changed since last month
This is the first edition. From next month, changes in the key figures are listed here automatically.
How accurate was the model compared with closing odds?Back-test
We re-ran our model week by week through 2022-23 to 2025-26, refitting it each Monday on data available before that week, and compared its probabilities with margin-free bookmaker closing odds for the same 18,807 matches. These are out-of-sample back-test predictions, not live picks.
Closing odds came out ahead. That is the expected result: closing prices aggregate team news, line-ups and the views of the whole market up to kick-off, while our model uses results only.
| Measure | Closing odds | Our model | Gap |
|---|---|---|---|
| 1X2 Brier score | 0.592 | 0.609 | 2.8% |
| 1X2 log loss | 0.991 | 1.016 | 2.5% |
| Over/Under 2.5 Brier score | 0.239 | 0.246 | 2.7% |
In which leagues was the model closest to the market?Back-test
Gap = how much higher the model's Brier score was than closing odds. Second-tier leagues are included because the model is trained on them; we do not publish predictions for them.
Embed this chart
| League | Matches | Closing Brier | Model Brier | Gap | Closing log loss | Model log loss |
|---|---|---|---|---|---|---|
| 2. Bundesliga | 1,176 | 0.620 | 0.630 | 1.7% | 1.032 | 1.048 |
| Championship | 2,153 | 0.621 | 0.634 | 2.0% | 1.033 | 1.053 |
| Eredivisie | 1,189 | 0.553 | 0.566 | 2.4% | 0.932 | 0.952 |
| Serie A | 1,520 | 0.577 | 0.592 | 2.5% | 0.969 | 0.991 |
| Ligue 1 | 1,297 | 0.583 | 0.598 | 2.6% | 0.978 | 1.000 |
| Primeira Liga | 1,180 | 0.529 | 0.543 | 2.6% | 0.899 | 0.922 |
| Segunda División | 1,788 | 0.619 | 0.636 | 2.7% | 1.029 | 1.055 |
| Ligue 2 | 1,331 | 0.625 | 0.643 | 2.8% | 1.040 | 1.067 |
| La Liga | 1,520 | 0.570 | 0.586 | 2.9% | 0.960 | 0.985 |
| Bundesliga | 1,224 | 0.576 | 0.594 | 3.1% | 0.970 | 0.997 |
| Brasileirão Série A | 1,463 | 0.594 | 0.614 | 3.3% | 0.996 | 1.024 |
| Premier League | 1,520 | 0.570 | 0.590 | 3.5% | 0.960 | 0.990 |
| Serie B | 1,446 | 0.628 | 0.653 | 4.0% | 1.044 | 1.080 |
Has the gap changed season by season?Back-test
Embed this chart
| Season | Matches | Closing Brier | Model Brier | Gap | O/U 2.5 closing | O/U 2.5 model |
|---|---|---|---|---|---|---|
| 2022-23 | 4,771 | 0.594 | 0.610 | 2.7% | 0.240 | 0.246 |
| 2023-24 | 4,732 | 0.589 | 0.606 | 2.8% | 0.236 | 0.243 |
| 2024-25 | 4,653 | 0.590 | 0.609 | 3.2% | 0.240 | 0.247 |
| 2025-26 | 4,651 | 0.595 | 0.610 | 2.4% | 0.242 | 0.248 |
Are the probabilities well calibrated?Back-test
Each home, draw and away probability is placed in a 10-point band; we then compare the average probability in the band with how often those outcomes happened. Points on the dashed line are perfectly calibrated.
Embed this chart
| Band | Model: predicted | Model: observed | Closing: predicted | Closing: observed |
|---|---|---|---|---|
| 0–10% | 7.2% | 6.6% | 7.5% | 4.8% |
| 10–20% | 16.2% | 15.7% | 16.0% | 14.3% |
| 20–30% | 25.6% | 26.3% | 26.0% | 25.5% |
| 30–40% | 34.3% | 33.1% | 33.9% | 33.9% |
| 40–50% | 44.6% | 44.1% | 44.6% | 45.9% |
| 50–60% | 54.4% | 54.9% | 54.6% | 56.1% |
| 60–70% | 64.2% | 65.3% | 64.5% | 66.9% |
| 70–80% | 74.3% | 76.7% | 74.5% | 78.8% |
| 80–90% | 83.9% | 84.0% | 83.7% | 87.8% |
What happened when the model and the market disagreed?Back-test
We count a disagreement when the model's and the market's probability for any outcome differ by 5 percentage points or more. That happened in 10,533 of 18,807 matches.
| Direction | Outcomes | Model avg | Market avg | Observed |
|---|---|---|---|---|
| Model higher than market | 9,930 | 39.5% | 30.4% | 28.2% |
| Market higher than model | 8,821 | 36.9% | 46.3% | 48.4% |
Per match: market gave the actual result the higher probability in 62.4% of disagreements; the model in 37.6%.
How does the live record compare?Live
The live graded record is building up; see the accuracy page for the latest figures.
Methodology
Walk-forward back-test: for every week from 2022-23 to 2025-26 we refitted our Dixon-Coles model using only matches played before that Monday, then predicted the week's matches. Closing odds (the market average just before kick-off) were converted to probabilities by dividing each implied probability by the book total (the proportional method), which removes the bookmaker margin. Accuracy is measured with the Brier score (squared error over home, draw and away; lower is better) and log loss.
Leagues: the nine club leagues we publish plus four second tiers (Segunda División, 2. Bundesliga, Serie B, Ligue 2) that the model trains on. Results are after 90 minutes.
Limitations
- A back-test is not a live record: the model's settings were chosen with knowledge of past data, so live performance can be slightly worse.
- Closing odds include information our model never sees (injuries, line-ups, market money), so beating them is not the aim; the comparison shows how far a results-only model sits from the market.
- Over/Under 2.5 closing odds are missing for the Brasileirão and some matches, so that comparison covers fewer matches.
Download the data
Aggregated study figures only (no raw bookmaker odds). Licensed CC BY 4.0; attribution required: “Source: ProbaPredict (probapredict.app)”.
Download the data (CSV)How to cite
ProbaPredict Data Desk (2026). "Our Model vs the Bookmakers: 18,807 Matches Tested". ProbaPredict. https://probapredict.app/studies/model-vs-bookmakers (accessed 2026-10-02). Licensed CC BY 4.0.
Historical results and closing odds: football-data.co.uk.
Previous editions
Related pages
- Our live accuracy record
- How accurate are football predictions?
- What are fair odds?
- Implied probability calculator
- How our model works
- Which Football League Is the Most Predictable?
- All football data studies
- Press and data
18+. These are statistical findings, not betting advice; nothing here suggests a way to beat bookmakers. Gambling can be addictive; only bet what you can afford to lose. Free help: BeGambleAware.org · GamStop.