Backtest: 2022 and 2024
The same model, run with only the data that existed a few days before each election, scored against the actual results.
2022
Headline
- Frozen as of
- 2022-11-05
- Seats called correctly
- 135 of 141 contested (95.7%)
- Brier score
- 0.034 (0 = perfect; 0.25 = coin flip)
- Log loss
- 0.133
- Margin error
- MAE 5.4 pts · RMSE 7.8 · bias +0.6 (positive = Dems beat forecast)
- 90% interval coverage
- 90% of actual margins fell inside the 90% range (target 90%)
- Statewide environment
- forecast D+0.9 vs actual contested-race aggregate D+1.4 (error -0.6)
- House Dem seats
- forecast 101.2 [94–109] · actual 102 · P(D majority) 48%
- Senate seats up
- forecast 12.9 [10–15] of 25 · actual 12
Inputs used: district leans from statewide races before the cycle; ballot (names/parties) and uncontested status; incumbency by name-match to previous winner; statewide polls up to the freeze date; 2022 legislative result (2024 backtest only). Not available point-in-time: district polls; external ratings; target lists; campaign finance (no point-in-time archive).
Calibration
| Forecast P(Dem) | Seats | Avg forecast | Observed Dem wins |
|---|---|---|---|
| 0.00-0.05 | 45 | 0% | 2% |
| 0.05-0.20 | 9 | 10% | 0% |
| 0.20-0.35 | 5 | 26% | 20% |
| 0.35-0.45 | 2 | 41% | 50% |
| 0.45-0.55 | 4 | 48% | 25% |
| 0.55-0.65 | 2 | 60% | 0% |
| 0.65-0.80 | 5 | 73% | 80% |
| 0.80-0.95 | 12 | 88% | 100% |
| 0.95-1.00 | 57 | 99% | 100% |
Misses (6)
| Seat | P(Dem) | Forecast | Actual | Democrat | Republican |
|---|---|---|---|---|---|
| H16 | 44% | R+1.1 | D+13.4 | Robert F Matzie | Rico T Elmore |
| H18 | 61% | D+2.2 | R+4.6 | Laurie Smith | Kathleen Christine Tomlinson |
| H72 | 0% | R+26.8 | D+9.1 | Frank Burns | H Renae Billow |
| H118 | 35% | R+3.0 | D+4.7 | James L Haddock | James E May Iv |
| H142 | 58% | D+1.6 | R+0.2 | Mark Moffa | Joseph F. Hogan |
| H160 | 68% | D+3.6 | R+4.7 | Catherine E Spahr | Wendell Craig Williams |
2024
Headline
- Frozen as of
- 2024-11-02
- Seats called correctly
- 120 of 125 contested (96.0%)
- Brier score
- 0.029 (0 = perfect; 0.25 = coin flip)
- Log loss
- 0.101
- Margin error
- MAE 5.3 pts · RMSE 6.4 · bias -4.3 (positive = Dems beat forecast)
- 90% interval coverage
- 97% of actual margins fell inside the 90% range (target 90%)
- Statewide environment
- forecast D+0.9 vs actual contested-race aggregate R+3.2 (error +4.1)
- House Dem seats
- forecast 103.1 [97–110] · actual 102 · P(D majority) 67%
- Senate seats up
- forecast 11.9 [10–14] of 25 · actual 10
Inputs used: district leans from statewide races before the cycle; ballot (names/parties) and uncontested status; incumbency by name-match to previous winner; statewide polls up to the freeze date; 2022 legislative result (2024 backtest only). Not available point-in-time: district polls; external ratings; target lists; campaign finance (no point-in-time archive).
Calibration
| Forecast P(Dem) | Seats | Avg forecast | Observed Dem wins |
|---|---|---|---|
| 0.00-0.05 | 49 | 1% | 0% |
| 0.05-0.20 | 11 | 13% | 9% |
| 0.20-0.35 | 1 | 21% | 0% |
| 0.35-0.45 | 3 | 39% | 0% |
| 0.55-0.65 | 2 | 59% | 0% |
| 0.65-0.80 | 3 | 74% | 67% |
| 0.80-0.95 | 10 | 89% | 90% |
| 0.95-1.00 | 46 | 99% | 100% |
Misses (5)
| Seat | P(Dem) | Forecast | Actual | Democrat | Republican |
|---|---|---|---|---|---|
| H72 | 15% | R+7.8 | D+2.8 | Frank Burns | Amy Marie Bradley |
| H142 | 62% | D+2.3 | R+9.6 | Anna K Payne | Joseph F Hogan Iv |
| S5 | 83% | D+7.6 | R+0.8 | James Dillon | Joseph Samuel Picozzi |
| S37 | 56% | D+1.2 | R+5.7 | Nicole Stoicovy-Ruscitto | Devlin J Robinson |
| S49 | 76% | D+5.4 | R+8.2 | Robert James Wertz Jr | Daniel J Laughlin |