PPFA Learns From Its Mistakes (Maybe)

Since starting this website in 2015, I’ve made my fair share of predictions. Some of them have been right. Some have been wrong.

With the midterms just eight days away, another set of predictions looms. Before I lock them in, I thought I might be able to learn from past mistakes and potentially implement those lessons into this year’s predictions.

So far this cycle, I’ve made predictions at the “six months out” mark, the “one month out” mark, and the “two weeks out” mark:

• Six Months Out: Republicans take the House. Republicans win 51 or 52 in the Senate
• One Month Out: Republicans take the House with 230 seats. The Senate goes 50-50 (Democrats keep “majority” with VP tiebreaker)
• Two Weeks Out: Republicans take the House with 235 seats. Republicans win 52 seats in Senate.

These predictions aren’t final. Next week, I’ll be calling upon all my knowledge to give you the best final predictions I can. To that end, today I’ll be reflecting on my past mistakes. I want to find what I was saying at points of past cycles, including what I said on the morning of the election, and then compare those predictions to the actual results. I’ll then ponder two general questions: 1) Where I was wrong? 2) Can I learn anything from those mistakes?

So let’s take a look…

(Author’s note: although I started this website in 2015, I didn’t make any Congressional predictions in the 2016 election. In those early days, I wrote about “presidential politics” and nothing else — hence the name of the website. Only later did I branch into history and other kinds of politics. Therefore, a 2016 postmortem isn’t terribly helpful, so we’ll start in 2018.)

---

2018

For the 2018 Midterms, here were my predictions at several points of the cycle, followed by the actual results of the election:

• Six Months Out prediction: Democrats take the House, Republicans keep Senate.
• One Month Out: Still Democrats take the House, Republicans keep Senate.
• Final prediction: Still Democrats take the House, 223-212. Republicans keep Senate at 50-50 plus VP Pence.
• Actual results: Democrats take House, 235-200. Republicans keep Senate, 53-47.

Takeaways:

• Although I correctly predicted the chamber majorities, I underestimated the margin for the winners.
• I’m fine with being off by 12 House races. There are just so many! And I don’t take the time to learn about individual districts. (You pay for what you get here at PPFA.)
• The Senate results hurt a bit, though. In my final Senate prediction column, I correctly predicted 32 of the 35 Senate races, including three flips: Arizona and Nevada flipping blue, North Dakota flipping red.
• However, all three PPFA losses were predictions for Democratic holds in states where Republicans won: Florida, Indiana, and Missouri. Ouch.

Now let’s juxtapose that performance with that of…

2020

In the presidential race, I correctly predicted 49/50 states, including the weird districts in Nebraska and Maine that cut against the rest of the state! The only miss was Georgia, which, with a margin of less than a quarter of a percent, was the closest result of the election. None of this has anything to do with Congressional predictions, but it’s my website and I can write whatever I want.

As for the 2020 race for Congress…

• Six Months Out: PPFA was on Covid hiatus, working to stimulate the slowing economy by eating takeout 3 to 4 times a week.
• One Month Out: In lieu of Congressional predictions, I instead explained that Biden would win the presidential election but that Trump would claim victory on election night like a cuckoo bananas dictator and wouldn’t accept the election results before pulling every lever within reach to overturn them. A++ job from PPFA.
• The House, Six Days Out: Democrats keep the House
• The Senate, Three Days Out: Democrats 50, Republicans 48, two run-offs in Georgia
• Final Predictions: Democrats win the House, 240-195. Senate is undecided (50D-48R-2 Georgia runoffs)
• Actual results: Democrats take the House, 222-213. Senate is undecided (50R-48D-2 Georgia runoffs)

House Takeaways:

This really ticked me off. Although I correctly predicted which party would win the House majority in both 2018 and 2020, the margins were off and in no perceivable pattern.

• In the 2018 House prediction, I underestimated the winner’s margin. In 2020, I overestimated it.
• That means, in 2018, I underestimated Democratic performance (slash overestimated Republican performance). In 2020, I overestimated Democratic performance (slash underestimated Republican performance).
• In sum: there was no trend in my House errors from 2018 to 2020, and therefore there’s nothing to learn from those mistakes.

This exercise has not helped so far and I’m regretting it.

Senate takeaways:

Ah, but in the Senate, there has been a pattern across the two data points.

• In 2018, remember, I had 32/35 Senate races correct, but all three misses were in the same direction: I picked Democrats, but Republicans won.
• In 2020, I did even better, picking 33/35 Senate races (if we include correctly predicting that both Georgia Senate races would go to run-offs). The two I missed were my “low confidence” Democratic pickups in Maine and North Carolina. Both went to Republicans.
• The pattern: I underestimated Republican results in the Senate both times.

This is actually kind of jarring. To have just five total Senate misses in 70 races across two elections is pretty good, but to have all five be wrong in the same direction is not so good. It is perhaps evidence of something systemically wrong with my analysis. If each prediction were a coin flip, the odds of all five going wrong in the same direction is 1/2 to the fifth power, or 1 in 32, about a 3% chance.

This mathematical improbability demanded that I look more into it, but there were no observable patterns in the cause of the misses. And trust me, I looked into it! Check out this useless Excel chart that took me like a half hour to make:

Aside from the Result column — which we already know contains the problem I’m trying to solve — are there any patterns? Incumbents won twice, challengers won three times. In the bottom three races, we can say that late momentum pointed toward the Republican win despite Democrats still leading a couple of those Election Day polling averages, but in the top two races Republicans both trailed in the polls and did not have momentum in the final month.

And yet, it’s not as if we “just can’t trust the polls.” It’s polling that helps me and others successfully predict so many Senate races, to say nothing of presidential election states. (Have I mentioned getting 49/50 right in 2020? Because I got 49/50 right in 2020.) I suppose we can’t “trust the polls” sometimes, but most times we can. The trick is finding out which is which. When does the exception push back against the rule? It’s easy to see in retrospect, but not in advance.

Anyway, these misses, however inexplicable, loom large over my upcoming predictions. There’s a good chance that if I determine a state or two as a tossup, this pattern might have me hedge toward Republicans.

But I’m not locking in any predictions yet. Here’s the upcoming schedule:

• Next up: a final deep dive into the House.
• Then, I’ll explore the possibility that the Democrats will shock us all.
• This weekend I’ll take my final deep dive into the Senate
• Monday, 11/7: Back by popular acclaim: your “Election Night Viewing Guide”!
• Tuesday, 11/8 (Election Day!): Final predictions

Five posts over the next week? You better believe it. ‘Tis the season.

---

Today’s featured image came from Auntie P at Flickr. I don’t know what the “P” stands for, but I like to imagine it’s Pasto.