The latest poll on North Carolina’s U.S. Senate race has Republican Thom Tillis (43 percent) and Democrat Kay Hagan (42 percent) neck-and-neck, with Libertarian Sean Haugh at 5 percent and 10 percent of likely voters still undecided.
The survey was taken by YouGov.com for The New York Times and CBS News. Within minutes of its release, Hagan partisans began discounting it on the grounds that YouGov.com relies on panels of online respondents rather than phone calls to produce its samples.
The tendency to discount polls based on new sampling methodology isn’t unique to Democrats. I’ve seen Republicans do it, too, when pollsters using robocalls produce survey results they don’t much like. While informed skepticism about all forms of polling is welcome, it is foolish to insist that the only valid political surveys are those taken by in-person telephone calls.
When it comes to election polls, all that really matters is their predictive value. As automated calls and Internet surveys have become more prevalent in the market (thanks to matters of cost and reliability), their results have become easy to compare with traditional live-questioner work. As far as I know, there isn’t appear to be any consistent pattern. Some traditional pollsters have an impressive record. Some have blown it, big time. You can say the same about the non-traditional ones.
Look at the pollster rankings Nate Silver produced for FiveThirtyEight after the 2012 presidential race. Of the 23 firms with a sufficient quantity of surveys, 12 used live callers, four used robocalls, six used Internet sampling, and one (Rasmussen) used a combination of robocalls and Internet. Half of the live callers were above the median, one (Marist) was at the median, and five were below it. Four of the seven Internet firms ranked in the top half. Robocallers fared worse in 2012, with only Survey USA in the top half (North Carolina’s own Public Policy Polling ranked 15th). On the other hand, robocall firms had a better year in 2010, which may be more relevant to this year since it was a midterm cycle.
Admittedly, online polling looks and feels a lot different than traditional polling. The latest Tillis-Hagan poll from YouGov.com is based on a sample of 2054 responses collected from August 18 to September 2. With a live-caller poll, the cost-per-completed-interview is significantly higher, so you typically get a smaller sample taken over two, three, or four days.
On the other hand, robocalls are relatively cheap and often produce significant samples in two days, and sometimes just one. I wouldn’t use it to probe public opinion about a wide range of complicated issues, mind you, but for election-matchup questions it can produce valid results when done carefully. (In recent cycles, Survey USA and Rasmussen represents the highs and lows of the practice, respectively.)
Here’s another reason why Hagan partisans shouldn’t have been criticizing the YouGov poll: its results are consistent with the other available data. Based on the likely voter polls to date, what we can say is that both candidates have support in the low 40s, with Haugh continuing to poll relatively well and a sizable number of voters still weighing their options. The race is still wide open. In most other Senate races around the country, even the tight ones, the both major-party candidates are polling two, three, even four points higher than Hagan and Tillis are. Because I don’t think Haugh will end up where he is now, I consider some of his vote to be undecided at the moment, too.
We’ll soon have more data on the race from PPP, Civitas, USA Today/Suffolk, and probably Survey USA (they often do polls for North Carolina TV stations). If you really want to know what’s happening, don’t pick and choose among them based on whether they do live interviews, robocalls, or Internet sampling. Judge them according to past performance, average the results across pollsters, or both. That’s what political pros and election-projection websites are doing, now. They aren’t finding excuses to filter the data based on what they wish to be true.
John Hood is president of the John Locke Foundation.