The Case of the Puzzling Polls

“The science of public surveying is in something of a crisis right now.” –Geoffrey Skelley, political analyst, University of Virginia’s Center for Politics.

One week out from the Iowa caucuses, the primaries might be more volatile than you think. Today and tomorrow, I’m explaining why. Today: to what extent do we trust recent polling?

Over the weekend, the New York Times‘s Frank Bruni wrote a piece skewering horse race coverage and the unreliability of polling. I’m guessing that he, like most of the civilized world (this blogger not excepted), clings to the idea that the combination of faults in polling strategy and our trust in them account for Trump’s ostensible lead in the GOP Primary. His premise must be considered. In Iowa, Cruz was up 2 in a poll completed on the 19th, but the very next day yielded a poll where Trump was up 11. In New Hampshire, two polls completed on the 18th had markedly different results for John Kasich; one had him at 20 points, the other at 6. The Democratic side, meanwhile, has not been immune to these elastic surveys. In Iowa, a poll had Hillary Clinton up 29 on the 18th, but another had Bernie Sanders up 8 just two days later. Bernmentum is one thing; a 37-point swing in 48 hours is something else entirely.

Many theories offer explanations for this erraticism. Bruni reminds us that cell phones, the increasingly regular mode of communication for Americans, are an untappable resource for robo- and auto-dialed pollsters (calling cell phones manually, which is allowed, is a tremendously more expensive and time-consuming polling process generally avoided by polling groups); those who still rely on landlines are not a reliable sample of Americans. For a visual of this trend, I pulled this image from a 2014 Huffington Post article, which illustrated that about 40 percent of Americans only use their cell phones for calls, while a growing majority at least mostly do:

He also notes people’s impatience with getting surveyed, citing a pollster for Washington Post and Survey Monkey who warned that the people who still participate in lengthy polling phone calls also do not represent an accurate cross-section of American voters. U.S. News tells us that fewer people are accepting pollster’s calls and participating in their surveys. It takes a certain disposition or desire to do so, and those types of people might gravitate toward certain types of candidates.

It’s hard to argue that these factors don’t taint polling, though I’d like to think that modern pollsters, at least good ones, are factoring in these hurdles when reaching out to voters (for example, making sure to find enough young people through landlines). Of course, the next step — figuring out what to do with this information in order to predict results — is difficult. Like Trump himself, we’re really good at pointing out problems, but clear solutions are hard to come by.

I perceive that most people think polling bias is hurting their own candidate. Sanders supporters see the cell phone issue as the most glaring example of polling bias since Readers Digest predicted Alf Landon over FDR in a landslide because their reader polls suggested as much. (Republicans were much more frequent Readers Digest subscribers.) Sanders’s base is young liberals, a demographic that’s almost exclusively cell-phoned (a word I just made up). Yet, we must also consider that Sanders’s supporters are more enthusiastic than Clinton’s (indeed, I know of no passionate Clinton supporter, though most of my Democratic friends are also fine with her); therefore, as the U.S. News report suggested, they’re more likely to patiently share their opinions on the race.

On the Republican side, if you’ll allowed my continued conjecture, polling bias can be spun any number of ways. Just as enthusiastic Bernieholics are more likely to welcome a pollster’s call, Trumpeters are just as fanatical, perhaps to the extent where, as their tactless leader mused this weekend, he could murder someone and still win. (I have to reserve one line for that Trump comment: such a remark speaks less to the loyalty of Trump’s supporters and more to how he perceives their character.) I’d also venture a guess that Trump’s base, notoriously less educated and presumably less modern than the supporters of other Republicans, is more likely to still be on landlines as well, further boosting his polling among pollsters’ sample sizes.

But again there’s a counterargument. Trump does even better in online polling than traditional cold calling — ten points higher (31-21) according to an October study. Why? One hypothesis is that his supporters, especially outside of the South and Midwest, are more sheepish about their allegiance and less likely to share their opinion with a person than a machine. In other words, Trump is so demonized by a majority of Americans that many Trump supporters have been quiet about their support, waiting to step out of the closet on primary day and into the safe confines of the voting booth.

Meanwhile, we’re also cursed with an old problem — trying to predict who will vote is not easy. Trump and Sanders have sizable enthusiasm gaps over their respective rivals, but not only does an enthusiastic vote still just count the same as an unenthusiastic vote — which is to say, it’s just one vote either way — but each camp should be worried about the reliability of their voter. Trump’s are less educated, and the less educated are less likely to vote than their diploma’d counterparts. (Maybe they’re worried there’s math involved?) Sanders’s supporters are young, and the young are less likely to vote than their elders. (Maybe they’re worried there’s a breathalizer?) Still, pollsters work to find predictive methods when testing the waters; some of them poll “registered voters,” others “likely voters,” and others use voters who voted four years ago. Some use a combination of these filters and others. But only on primary and caucus days will we truly know who shows up.

And the first is only seven. days. AWAY!

Tomorrow, I’ll focus in on another contributor to these puzzling polls.