The first and only time I ever helped take a political survey was during my undergraduate days at the University of North Carolina-Chapel Hill back in the 1980s. I was one of many journalism students who made phone calls on behalf of the Carolina Poll, a regular survey of North Carolinians on political and social topics.
During the ensuing three decades, the polling industry underwent dramatic growth and change. Here in North Carolina, several different groups now commission or conduct public-opinion surveys on a regular basis. During election season, other out-of-state pollsters swoop in to poll voters, either on their own dime or in partnership with local media outlets.
As it turns out, however, the idea of basing statewide polls around college students conducting live interviews is far from outdated. According to a new study by Nate Silver’s indispensable FiveThirtyEight.com website, in fact, the polls produced by North Carolina universities are at least as reliable as other surveys, and sometimes better.
For the site’s rankings, each pollster is evaluated on the basis of methodology, bias, and accuracy. Generally speaking, the closer a final election prediction is to the actual election result, the higher the pollster ranks.
State political junkies will find the rankings of five pollsters to be particularly interesting. Public Policy Polling (PPP) is a Democratic-leaning company based in Raleigh that surveys voters by using automated calls. National Research is a Republican-leaning company that conducts live-operator polls for the Raleigh-based Civitas Institute, a conservative think tank. Survey USA is an automated-call pollster that also does election surveys for Civitas. And two Triad-area institutions, High Point University and Elon University, conduct regular live-operator polls using trained students.
On the FiveThirtyEight.com letter grading system, Survey USA is one of the few pollsters to get an A. Elon University gets a B. High Point University, PPP, and National Research get B-minuses.
While these are the pollsters that produce the greatest amount of survey research on North Carolina issues and opinions, the Senate race between Kay Hagan and Thom Tillis has certainly drawn lots of other pollsters into the state this year. Among the ones you probably see or hear about most often, the highest-ranked are ABC News/Washington Post, CNN/Opinion Research, and NBC News/Wall Street Journal, all of which get A-minuses in the pollster rankings. Mason-Dixon, Fox News, Pew Research Center, and CBS/New York Times also compare fairly well, getting Bs. Less impressive in recent election cycles have been surveys from Gallup (C+), Rasmussen Reports (C), Gravis Marketing (C), and Harris Interactive (D+).
Even the most accurate election pollsters produce some screwy samples or weight them unwisely. In addition to ranking them by accuracy, FiveThirtyEight.com also provides estimates of the average partisan bias. For top-flight pollsters, the bias is often very small. In recent elections, for example, Survey USA’s predictions have been slightly (0.2 percent) more Republican than electoral outcomes. So have predictions by National Research (+0.5 percent Republican) and — get this — PPP (+0.7 percent Republican). The Elon University poll, on the other hand, has leaned slightly Democratic (0.2 percent) while the High Point University poll has been among the few with no partisan bias at all.
During the final weeks of the 2014 election cycle, most of these pollsters will produce additional surveys on the Tillis-Hagan race and others. If you want to know what’s really going on in North Carolina politics, you won’t discount any of them. I keep a rolling average of all statewide polls, along with the dates they were taken and other information with which to compare and contrast. Trusting only polls from groups you like (selection bias) or that produce results you like (confirmation bias) will lead you astray more often than not.
And when it comes to predicting the outcomes of elections, don’t fall into the trap of assuming that only paid professionals staffing phone banks or running auto-dialing machines produce valid results. The surveys that Elon and High Point conduct are filled with useful insights about what North Carolinians think — and how they are likely to act on it when casting their ballots.
John Hood is chairman of the John Locke Foundation.