RALEIGH – Sitting through a presentation last week about seemingly high average test scores in one of North Carolina’s larger school districts, it occurred to me that not only most of the audience but also the district’s administrators actually believed the scores were meaningful. It was a scary thought.
When a North Carolina district reports that, say, 90 percent or more of its students tested “at grade level” in reading and math, it sounds impressive. Unfortunately, it doesn’t come close to meaning what the average person might think it means. It doesn’t signify that only about 10 percent of students lacked competence in the basic skills and subjects associated with a given academic grade. Instead, it simply means that 90 percent of students met the shockingly low standards set for public schools by lawmakers and policymakers in Raleigh. Many, and often most, of these “passing” students did not master the material they need to know, according to more rigorous national and international standards.
The gap between independent assessments and state test scores is large just about everywhere in the United States, but North Carolina’s record is particularly embarrassing in this regard. Much like the graduation-rate scandal of a couple of years ago – when North Carolina was caught manufacturing what can only be called a fraudulent statistic about high-school completion, thus hiding the true extent of the problem – our state’s testing program does not communicate complete and accurate information to parents and taxpayers. It may make people feel better but it is not a reliable guide for formulating or evaluating educational policies.
In Monday’s Wall Street Journal, Checker Finn and Diane Ravitch wrote about the gap between national and state assessments in a piece (unfortunately not online) entitled “Basic Instincts.” Both wrote from personal experience with the issue, having served on the National Assessment Governing Board, which oversees the National Assessment of Educational Progress (NAEP). While hardly a perfect system, NAEP is essentially the best and most reliable yardstick for tracking trends in educational performance. And, in fairness to North Carolina, the NAEP data since the late 1980s do show our state making solid gains in performance, though not nearly as dramatic a gain as our own state test scores would seem to indicate.
For North Carolina education-watchers, the most eye-opening part of the Finn & Ravitch piece is a table comparing proficiency scores on state and national tests in reading and math. Only Tennessee looks like a worse offender than North Carolina by this measure. Tennessee’s reading proficiency was 88 percent on state tests and 26 percent on the NAEP. North Carolina’s was 88 percent and 27 percent, respectively. With regard to math, Tennessee’s gap was 87 percent vs. 21 percent and North Carolina’s was 84 percent vs. 32 percent.
Years ago, having quantified the same problem, JLF analysts found that while both the NAEP and North Carolina’s end-of-grade (EOG) tests reported four performance levels, they were out of kilter by about a full level. That is, performance that was considered proficient or Level 3 on the NAEP – meaning that a student could demonstrate solid but not advanced performance across a range of grade-level expectations – was rated as Level 4 on the EOG. Likewise, what the NAEP merely called “basic” or Level 2 – an incomplete understanding of the material, leaving the student only “minimally qualified” to study at the next grade level – was nearly equivalent to proficiency or Level 3 on the EOGs. And what the NAEP labeled “advanced” was a level of academic performance that, unfortunately, North Carolina’s testing system does not really measure at all.
The next time you hear North Carolina politicians or education officials waxing eloquent about three-quarters or more of a school’s students being “at grade level,” keep in mind that, whether they realize it or not, they are engaging in rhetorical excess. At that same school, it is likely that fewer than half the students can actually read, write, and compute proficiently at their grade level.
The academic performance is still probably better than it was a decade ago, at least in North Carolina. But it’s not exactly awe-inspiring.
Hood is president of the John Locke Foundation.