A study released recently by the John William Pope Center for Higher Education Policy shows how the annual ranking of American colleges and universities by the magazine U.S. News and World Report is an unreliable guide to academic excellence.

The report, “Do College Rankings Mean Anything” (Inquiry No. 17, Aug. 23, 2004), was written by George Leef, executive director of the Pope Center, and Michael Lowrey, and associate editor of Carolina Journal and a former college instructor.

U.S. News released its annual survey results Aug. 20. Duke University, Davidson College, Wake Forest University, and the University of North Carolina at Chapel Hill were all highly rated.

In their report, Leef and Lowrey find that the criteria used by U.S. News in its much-followed ranking system are at best weak proxies for academic strength. “Schools with high rankings don’t necessarily provide a strong educational experience for their students, and schools with relatively low rankings may offer students better instruction in a more solid curriculum,” Leef said.

Analyzing each of the factors that go into the U.S. News ranking system, Leef and Lowrey conclude that they have little or no bearing on educational quality. U.S. News relies on six factors, they report, all six of which are “either input measures or subjective evaluations” given weights by the magazine that are “entirely subjective.”

For example, one measure that counts for “[f]ully one-fourth” of the U.S. News ranking is “academic reputation.” It asks the top three officials at each university to rank schools similar to their own. The problem is, as Leef and Lowrey said, “[t]he officers who are surveyed are very knowledgeable about their own institutions, but in most instances they would have little or no direct knowledge about the academic environment even at other schools in their own state, much less around the entire nation.” Nor does academic reputation necessarily correlate with student learning, they write.

Another factor in the U.S. News rankings is faculty compensation, in which the higher the compensation, the higher the school’s score will be. But “[h]igh pay doesn’t guarantee teaching excellence,” Lowrey said. “In fact, some ‘superstar’ professors with six-figure salaries are notorious for neglecting their students.”

The authors say there is no ideal ranking system and advise students and parents to carefully evaluate the curriculum and faculty at schools and departments to gauge how strong or weak the educational experience is likely to be. The report is online at the Pope Center website.

Jon Sanders is assistant editor of Carolina Journal.