Worried pollsters admit huge problem
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Eniola Odetunde/Axios
Four years later, it’s still easy to hate on the polls. Even if they called the presidential race "right" this time, the national polls vastly underestimated support for President Trump once again, leading to a major industry soul searching about how to fix some fundamental problems.
Why it matters: Without data to provide a beacon for elections, there isn’t much else to go on.
- With a sharply divided public consuming increasingly partisan-skewed information, America has depended on data and statistics to provide clarity and reasonable expectations going into elections.
- Trust in the numbers has largely been eroded, and now they're more subject to partisan attack and spinning.
- They're even under attack from Trump, who baselessly claimed that "fake polls" showing Joe Biden with a big lead were "designed to keep our voters at home, create the illusion of momentum for Biden, and diminish Republicans' ability to raise funds."
Be smart: Pollsters privately acknowledge the huge problem, but publicly say they need the final vote tallies to begin the autopsy. Biden is currently winning by 3% of the popular vote, and an average of national polls gave him a 8.4% lead.
- Conventional political wisdom that the Democrats would take over power in the Senate also was based on faulty polls that missed down ballot race projections, too.
- “Let’s pump the brakes a bit” before we indict the whole industry, said Courtney Kennedy, director of survey research at Pew Research Center and an author of the industry’s 2016 autopsy report.
Forget the “shy Trump voter” narrative. Before the election, pollsters were worried that perhaps some Trump voters surveyed were simply too embarrassed to tell people that they were voting for him.
- The real problem is even worse. Virtually every poll did not survey enough Trump supporters, period — meaning there’s a huge sampling error the industry needs to reckon with.
“We’ve been struggling with this issue and we keep trying to fix it and it’s not totally working,” said one worried pollster who did not want to be named for fear of ruining his industry.
- It’s called “non response,” meaning pollsters are not getting enough Trump voters to even participate in a survey and answer questions.
- This isn’t just a phone call survey problem. It's in online surveys and text surveys, too.
- “The major problem, the fundamental issue in the polling industry is declining response rate,” said Jon Cohen, chief research officer at SurveyMonkey.
- Another twist on this is not capturing enough “non-college educated voters” who were missing from 2016 surveys. The 2020 election proved Trump expanded his voter turnout in new demographic groups, and still polls missed them.
Another continuing culprit: state polls. As we have written before, poorly conducted state surveys were to blame for misleading the public. Biden managed to flip Michigan, Wisconsin, Arizona and Pennsylvania, but polls still overestimated enthusiasm for Biden in those states and others that were considered on the bubble.
- “Before Election Day, it seemed to me that, sure, some polls might not have enough Trump supporters but that was okay if a poll was statistically adjusted properly to make it representative,” said Kennedy.
- “The question looming now is whether that’s true. It’s not clear that the polls checked all the boxes and had good statistical adjustments and whether they still worked.”
The bottom line: It may be months before we learn why the polls failed to meet the public’s expectations in 2020, but two clear problem areas have been problems for a long time.
