A state audit finding potentially “hundreds of millions of dollars” of Medicaid mismanagement also called for “a scientifically valid study” of North Carolina’s showcase Medicaid program due to suspect savings claims and questionable methodologies used to evaluate it.

“The question should arise, if CCNC [Community Care of North Carolina] saves significantly on Medicaid expenditures, why does North Carolina spend so much more on Medicaid than comparable states?” the audit asked.

The CCNC program is a nonprofit agency contracted to administer the state’s Medicaid program. It enrolls 1.2 million of the state’s 1.5 million eligible Medicaid clients, and some 64 percent of its aged, blind, and disabled population.

Suspicion about the program’s claims of enormous savings through its innovative “medical home model” of treatment has emerged among leading national experts, whose warnings have been ridiculed by some state lawmakers and Medicaid officials.

Yet the audit, released by state Auditor Beth Wood Thursday, validates critics’ concerns. One audit recommendation is to “engage medical researchers to perform a scientifically valid study based on actual data to determine whether the CCNC model saves money and improves health outcomes.”

It said “more than a decade of data exists that would allow a study by medical researchers on whether the medical home model truly saves money and/or results in better medical outcomes.”

The audit said that an actuarial study by Milliman Inc. finding savings through CCNC “may be actuarially sound, [but] it does not provide the same quality of data that could be derived from medical research.”

Because other states are considering adopting CCNC-style programs based on claims of gigantic savings, “It would be a service to the nation as well as North Carolina to use its data to genuinely evaluate the questions associated with medical homes,” the audit said.

Al Lewis, one of the nation’s leading care management outcomes analysts, is among the experts who said the same thing long before the audit came out.

His book, Why Nobody Believes the Numbers, contains an entire chapter about “impossible results” being claimed by CCNC’s consultants. He also wrote a peer-reviewed article in the August edition of The American Journal of Managed Care questioning CCNC savings.

The audit “vindicated me, not that I needed vindication, because it’s simple arithmetic. Somebody eventually was going to figure out the arithmetic that wasn’t me,” Lewis said.

“Like the Mercer and TREO [consultants’] numbers before them, the Milliman numbers are completely invalid,” Lewis said.

“I’m not sure whether [Milliman] ignored the online federal data that completely invalidates their findings out of ignorance or by design, but either way, I’d recommend an apology, a retraction and a refund” of their consultant fees, he said. “By justifying the unjustifiable and keeping it funded another two years, that $500,000 consulting report cost state and federal taxpayers close to a billion dollars.”

The consultants used methodologies that incorporated “wacky trend assumptions,” Lewis said. “Wildly” overstating what health trends would have been without CCNC intervention resulted in “hugely inflating the savings.”

“Treo Solutions disagrees with that characterization of our study,” said Rich Keller, senior healthcare consultant at that firm. The other consultant firms also stand by their conclusions, and officials at CCNC have defended their operations.

A lengthy, written statement provided by CCNC in lieu of a requested interview said the program “has been the subject of highly detailed evaluations” and called its consultants “three of the nation’s premier actuarial firms.”

“Using data from different periods and several distinct methodologies [the Milliman report alone examines savings through three separate research methodologies], all three firms reached the same conclusion: CCNC provides North Carolina with tremendous return on investment,” the statement said.

Milliman found $984 million in savings from 2007-10. Treo Solutions said the savings was $1.5 billion from 2007-09. Mercer found between $708 million and $758 million from 2005-09.

Lewis pioneered disease management, one of the country’s most popular health care disciplines. He said if he were doing the analysis and “trying to portray the results in their best light, I might be able to find a break-even” conclusion.

Those diverse findings illustrate a divisive gulf in modern health care assessment. Traditional actuarial studies like those done for the state may produce vastly different results versus those found through the field of epidemiology, which relies heavily on biostatistics, or just simple plausibility testing.

Among weaknesses of the state’s consultant studies, in his view, is none was peer reviewed, nor did any compare performance with comparable states, both standard industry seals of validation.

Under CCNC’s patient-centered medical home system, a primary care physician is assigned to each patient to ensure delivery of consistent treatment, medication, and tests to improve poor health conditions. The goal is to better manage health problems and reduce more expensive hospitalizations and emergency room visits.

“The patient-centered medical home model simply raises costs, period,” Lewis said.

Among other defects Lewis found in the state’s studies are findings that are “blatantly contradicted by federal data that’s available for the world to see online,” he said.

The state’s studies fail even to mention the federal data, which show, among other details, neighboring states are spending less on their Medicaid programs, he said.

Lewis said the Medicaid Commission Report to Congress on June 11, 2011, shows North Carolina per capita costs were 24 percent to 40 percent higher for children and adults than they were in South Carolina, Georgia, Tennessee, and Virginia.

Given those trends, it should follow that care for the aged also would cost more in North Carolina than surrounding states, but it actually costs less. Lewis said that is because Medicare, not CCNC, pays for the aged, and adds validity to his claim CCNC costs too much.

Lewis also points to data from the federal Healthcare Utilization and Quality Project that North Carolina Medicaid hospital admissions remained nearly unchanged from 2000-09, despite CCNC’s charge to lower them. South Carolina’s performance through a standard, low-cost, managed care model matched or did better than North Carolina’s.

HCUP data for North Carolina shows from 2000-09, asthma admissions declined only 29.5 percent in the Medicaid population, compared to 32.2 percent among those not in Medicaid, and 47.7 percent among South Carolina’s Medicaid recipients.

During those same years, diabetes admission rates per 1,000 went up more in North Carolina’s Medicaid population (7.9 percent) than in the non-Medicaid population (3.9 percent). In South Carolina, the Medicaid admissions actually declined 7.4 percent.

But CCNC fired back at “the lack of rigor and intellectual dishonesty that characterizes Mr. Lewis’ repeated claims.”

In its written statement, it said: “HCUP and MACPAC cited by Mr. Lewis are insufficient for that analysis, because CCNC enrollment is not identified in those data,” and using mingled numbers would skew results.

“We got the actual claims data from the state of North Carolina so we got what we thought was the gold standard,” said Bob Cosway, one of the Milliman study principals.

When using the federal databases, “There’s potential that something gets lost in the way it’s made comparable” on a state-to-state basis because there are many variables in each state’s programs, Cosway said.

However, Carol Stocks, a spokeswoman at the Agency for Healthcare Research and Quality Center for Delivery, a federal-state initiative that sponsors the HCUP research, said its data is respected universally.

“We tend to be the standard that other people use to validate what they do,” she said. HCUP data come directly from the 46 participating states, and cover 97 percent of all U.S. hospital admissions.

“Researchers use it to create studies used in peer-reviewed journals, and they have pretty exacting standards,” Stocks said. “It’s also used by different state organizations … for their own reports in terms of cost, quality, and utilization of hospital care,” among other reasons.

Dan E. Way (@danway_carolina) is an associate editor of Carolina Journal.