By Jack O’Toole, Capitol bureau | With a presidential election that both sides are calling one of the most momentous in American history now only days away, partisans of Team Red and Team Blue are scouring polling data for even the slightest hint their side is going to win.
But given the closeness of this year’s race between Vice President Kamala Harris, a Democrat, and former GOP President Donald Trump, pollsters say such scrutiny is a fool’s errand.
“When every poll is this tight – when every poll is within the margin of error – that’s why we use the phrase ‘toss-up,’” Winthrop University pollster and political scientist Scott Huffmon told the City Paper in an Oct. 21 interview.
Focus on the right things
What’s more, he said, many voters are looking at the wrong polls anyway.
“The thing that matters most is, stop obsessing about the national polls,” Huffmon said. “National polls may show a trend, but because of the way the Electoral College works, it really just comes down to about seven states, so pay attention to quality polls from those battleground states.”
In 2024, the seven swing states – that is, the states that are actually thought to be competitive – are Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin.
But at a moment when political misinformation, including biased polling data, is rampant on the internet and social media, how can news consumers distinguish between “quality polls” from those states and the junk polls that keep creeping into their feeds?
According to Huffmon, this is a case where traditional, or “mainstream,” media brands deserve the highest level of trust – because accuracy is key to their credibility.
“Any legitimate national media, whether they lean right or lean left, want to get the results correct,” Huffmon said, pointing to outlets like The New York Times, CNN and Fox News, whose data unit was the first to correctly call Arizona for Biden on election night in 2020. “And because of that, because they want to be right, they’re not going to bias their polls.”
Beyond that, experts say, there are poll aggregators – websites like FiveThirtyEight and Real Clear Politics – that combine the best polls into an average result that should, in theory, reduce the risk of major error.
But even after you’ve found quality polls from reliable sources, experts say it helps to know a bit about how they report their results if you want to read one like a pro.
Polling 101
These days, most people who care about politics have at least a rudimentary understanding of how polling works: Contact a representative sample of Americans – say a thousand people – ask them how they’re going to vote, apply a little statistical magic and report the results.
If you do it right, you should end up with a snapshot of public opinion at the current moment that’s accurate within a specified margin of error – plus or minus three points, for instance, with a sample size of 1,000 people – to a 95% degree of confidence. In other words, with a sample that’s that big, there’s a 95% chance the results are within three points of what the poll says.
But pollsters say it’s important for news consumers to understand exactly what those terms mean before they start drawing conclusions – mainly because there’s leeway built in
First, they say, a snapshot is just that – a picture of public opinion at a specific moment in time. Today’s undecided voters are tomorrow’s margin of victory for one campaign or the other.
Second, the margin of error doesn’t apply to the spread between the candidates. Rather, it applies to every number in the survey. For example, if Candidate A is at 52% and Candidate B is at 48% in a poll with a three point margin of error, Candidate A could be ahead by as many as 10 points or behind by one. In other words – and somewhat counterintuitively – a four-point race with a three-point margin of error is, in fact, within the margin.
Moreover, the margin of error is not consistent across all results. For example, a typical poll of 1,000 Americans would include about 150 Black men and women, meaning that results from that subgroup, or crosstab, have a greater margin of error of about 10 points because the subgroup was a smaller number of respondents.
Finally, the 95% confidence level tells poll readers that 5% of the time, the poll will not be within the margin of error. So five out of every 100 polls conducted is likely to be wrong.
Trump and the challenges of modern polling
Even before Trump burst on the scene in 2015, political pollsters were struggling to find and survey representative samples of voters.
“There used to be two ways to reach people – call them on the phone or get them to answer a written survey,” South Carolina-based pollster Carey Crantford told the City Paper. “But those have now been eclipsed by cell phones and texts and emails, and that’s impacted electoral surveys significantly.”
Put simply, it’s harder to reach people than it was 15 or 20 years ago, when all you had to do was call their land line. And even worse, the people pollsters can reach most consistently using these newer methods aren’t necessarily fully representative of the population.
As a result, the polls in both 2016 and 2020 were off by some degree, especially in the swing states, underestimating Trump’s strength in both elections.
“The biggest issue was the underrepresentation of white male and non-college voters,” Crantford said, noting that these were two key Trump constituencies. “So pollsters have been trying to [account for that] by modeling their sample [and] tying the results of the future to the results of the past.”
In other words, most pollsters this cycle are massaging, or “weighting,” their results to make their sample look more like the one that actually turned out to vote in 2020.
And according to Crantford and Winthrop’s Huffmon, that’s the single biggest question with regard to current polling on the race: Is weighting results by so-called “recalled vote” a real solution to pollsters’ longstanding problems measuring Trump’s support, or is it causing them to miss changes in the makeup of this year’s electorate?
“For example,” Huffmon said, “if suburban women change their vote because of Roe v. Wade being overturned, that’s not going to show up as strongly as it should when you weight by previous vote.”
But in the end, no matter how hard they try to get it right, pollsters say you can’t be sure about anything when Harris and Trump are locked in a virtual dead heat in nearly every poll.
“We may be asking too much of polling these days,” Crantford said. “The utility of the tool is really in what you use it for.”
- Have a comment? Send to: feedback@statehousereport.com.
We Can Do Better, South Carolina!