Model calibration · back-test
Premier League.
For each probability band the model produces, what fraction of outcomes actually occurred? A well-calibrated model has the bars hugging the diagonal — meaning when it says "65%", the event hits ~65% of the time.
10-20%
Δ -8.4
20-30%
Δ +2.0
30-40%
Δ -3.0
40-50%
Δ -4.8
50-60%
Δ +9.2
60-70%
Δ +36.5
Expected (model)Actual hit rate322 matches × 3 outcomes
| Probability bucket | N outcomes | Hits | Expected | Actual hit rate | Δ (actual − expected) |
|---|---|---|---|---|---|
| 10-20% | 12 | 1 | 16.7% | 8.3% | -8.4 pp |
| 20-30% | 453 | 121 | 24.7% | 26.7% | +2.0 pp |
| 30-40% | 234 | 75 | 35.1% | 32.1% | -3.0 pp |
| 40-50% | 194 | 77 | 44.5% | 39.7% | -4.8 pp |
| 50-60% | 67 | 42 | 53.5% | 62.7% | +9.2 pp |
| 60-70% | 6 | 6 | 63.5% | 100.0% | +36.5 pp |