- "In the end the whole point of doing it is so that you get a sample that is representative of the electorate,” said Brock McCleary, vice president of Cygnal Polling and Analytics
In 2020, polls projected North Carolina and Florida to go for Joe Biden in the presidential race, but both ended up in Donald Trump’s electoral column. This year, polls show tightening races across the country, many within the margin of error, leading some to speculate whether poll accuracy is an issue again.
“Republicans in North Carolina were very confident about taking supermajorities in both chambers of the General Assembly earlier this year, but became concerned as polling numbers tightened over the summer.” said Andy Jackson, director of the Civitas Center for Public Integrity at the John Locke Foundation, “Some Republicans may be taking comfort in the fact that pollsters have overestimated Democratic support in past polls, with surveys famously underestimating Donald Trump’s strength in 2016 and 2020.”
In 2020, Real Clear Politics polls showed an average lead of 2.6 percentage points for Democrat Cal Cunningham in the race for U.S. Senate, but Republican Thom Tillis ended up winning by 1.8 percentage points.
“Polls had Roy Cooper up by an average of 11 percentage points, but he ended up winning by a comparatively modest 4.4%. Cooper also underperformed his polling numbers in 2016,” said Jackson.
Brock McCleary, vice president of Cygnal Polling and Analytics, says his organization has identified where polling errors can occur, and among the potential problems is when the method of reaching respondents is too narrow.
“If you’re not using email, panels, SMS [texts], and a whole combination of polls, then you don’t have a chance of getting a representative sample any more,” said McCleary. “A lot of it is diverse multivote data collection. A lot of the world is still operating as though it’s 1998. People are using a lot of landlines, and they’re answering their cell phones from numbers they don’t know. Refusing to update methodology to compute with the fact that people have cell phones but calling isn’t even the thing that they use it for. Calling is the lowest use of mobile devices. [People use it for] texting, email, browsing; somewhere low on the list is calling.”
Inaccurate polling issues can also arise when pollsters aren’t getting a representative population sample.
“The stakes of 2016 were largely about the media not anticipating the Trump coalition,” said McCleary. “They dramatically oversampled voters who have a bachelor’s degree or higher. Voters with a bachelor’s degree or higher are far easier to get on the phone and are far more eager to take a poll.”
He said before Trump, pollsters didn’t look at that. They had samples that weren’t representative of the electorate in terms of their educational attainment.
Along with the issues of getting working-class Republican voters to answer the phone, McCleary said there’s a trend of Republicans being shy to answer polls because of controversial cultural issues.
“It’s what we call social desirability bias,” he said. “It is not socially desirable to state that you are a center-right person. Particularly, if you are a Trump supporter, it is even more socially undesirable. That is largely the intention of the left, but it is important because it affects response rates among Republicans.”
McCleary has what he calls a sophisticated way of identifying states and media markets where there was likely to be a higher concentration of shy Trump voters. He did that because he wanted to know where there might be the greatest opportunity for gains.
Polling errors cause more than surprises on Election Day; they affect voter confidence and influence campaign spending. A Real Clear Public Affairs article says that polling can cause political parties to overfund candidates who they believe are in trouble and defund candidates who they think are too far behind. Inaccurate polling numbers can lead the public to believe in or lose faith in a candidate.
“We use SMS, and we do things a little differently than others to give Republicans assurance that this isn’t a media poll, but also because they’re working-class voters, they’re not going to talk on a live call with you for 15 minutes and tell you all of their political views,” said McCleary. “They’re busy, they’re working, and they’re not at their home office like most of white-collar America was post-pandemic.”