RALEIGH – North Carolina political and education leaders held a news conference Thursday to release state test scores and accountability outcomes for the 2003-04 school year. This has become an annual event in Raleigh, usually in August or early September, as the official ABCs of Public Education results are released for each public-school district in the state (some of which typically let preliminary results trickle out if they look particularly good).
In the past, my staff and I have paid a great deal of attention to this announcement. Starting just a couple of years after the program started in the mid-1990s, we began issuing our own statewide report card that used the ABC data and other state statistics to compute letter grades for each district linked to reasonable expectations about what a “successful” school would look like.
But this year, I’ve had a hard time working up any interest in the data release. It’s not that I don’t think educational accountability is a critical issue – there are few of equivalent importance in North Carolina politics – but I simply put little credibility in the state’s end-of-grade and end-of-course tests.
The basic problem is that every year we’re told how far our students have jumped, but the judges won’t let us see the measuring tape. We have no idea whether last year’s foot is the same length as this year’s foot. Indeed, what we do know about the process of devising and scoring North Carolina’s tests suggests that consistent measurement over time is probably impossible. Unlike many other states, which use independent nationally referenced tests, North Carolina generates its own tests and estimates the proper “cut score” – the percentage of questions a student must answer to be considered “at grade level.” Given that many of these cut scores are well below 50 percent, and sometimes approach one-third of the questions correctly answered, students well-coached in test-taking techniques have a good chance of passing based only on half-educated guesswork.
The argument for North Carolina-only tests is that they more closely track our state’s chosen curriculum, its “standard course of study.” But how different is our state’s chosen content from that of the rest of the nation? Should it even be substantially different, given the interrelated nature of our economy and society? More importantly, this possible advantage does not outweigh the disadvantages of 1) lack of comparability to other jurisdictions, 2) lack of confidence in the test’s independence and reliability, and 3) lack of access to the test questions (other states release old tests for public review, but North Carolina says it wants to save money by reusing the old tests and therefore hides them from view).
Apparently the 2003-04 results showed a slight increase in students scoring at grade-level, a decline in public schools meeting or exceeding expected growth, and an increase in schools meeting the No Child Left Behind Act’s federal growth standard. But this is all fruit from a poison tree. Also, it demonstrates that scoring changes matter more than academic performance (the gains on the federal measures were largely due to a statistical adjustment to allow more student groups to fall within a margin of error).
I’m not alone in my disaffection. These announcements clearly do not get the media and public attention it used to, and for good reason. As political candidates talk about education this fall, I’d like to see them discuss the possibility of adopting independent, reliable, and meaningful national tests. Armed with the results, we could rejuvenate a debate over education reform in North Carolina that has become confused and convoluted.
Hood is president of the John Locke Foundation and publisher of Carolina Journal.