Measuring the performance of colleges — and holding them accountable for their performance — is an unsolved problem. A new “earnings gains” measure of community colleges in North Carolina aims to solve it.
The state is increasing its focus on “performance funding” of the North Carolina Community College System. Performance funding combines measurement and accountability functions, using a set of goals that public colleges are expected to meet to determine in part how much funding they will receive from the General Assembly.
The NCCCS is trying to add a new measure to its performance-funding model based on graduates’ incomes. The “earnings gains” measure was approved on Aug. 21 by the State Board of Community Colleges, but still needs the legislature’s approval.
The metric, using wage records from the North Carolina Department of Commerce, measures the graduates of vocational programs who have held full-time jobs for the two years before enrolling and for the two years after graduation.
Accountability standards have evolved over the years. In 1998, the General Assembly required the State Board of Community Colleges to review its performance measures and implement a performance-based funding plan. Under the original plan, colleges that met all of the measures received bonus funding. Colleges that underperformed merely had to submit an action plan to the board, but were not penalized further.
In 2013, the General Assembly changed the structure so that colleges would be funded proportionately by how well their students do.
The legislature set aside $24 million for performance-based funding out of a $1.3 billion NCCCS budget in 2014-15. There are eight funding measures designed to apply to the various types of community colleges students; they range from GED passing rates and remedial student success to certification test passing rates and performance of transfer students.
One measure of performance missing from the list: an employment measure. That was not always the case. Originally, there were 12 measures, two of which were directly related to employment: “employer satisfaction with graduates” and “employment status of graduates.”
However, both measures were unreliable, according to Bill Schneider, the community college system’s vice president for research and performance management. He told the Pope Center that both measures had unrealistically high success rates, as they failed to include many people who were not working, used inconsistent survey methods, and cherry-picked which employers to survey.
While several states utilize some kind of employment metric, most do not. Even fewer have the measure that North Carolina settled on. A national higher education metrics expert with whom the NCCCS consulted, Nate Johnson of Postsecondary Analytics, told the Pope Center that earnings gains are a good measure and that by using it, North Carolina is “putting itself at the leading edge” of higher education systems seeking to measure workforce data. For example, he said, Florida simply provides average wages without any context or added value.
But he also said that measuring work force outcomes is “not a well-developed science yet in higher education.”
The average age of students measured using North Carolina’s metric is 34, six years older than the average of the community college system. Among the groups excluded from the measure are students in academic programs and students who are part-time workers, unemployed, or work in another state two years before or after community college.
Stanly Community College president Brenda Kays, who headed the committee that produced the measure, admitted that it is not perfect and may need to be adjusted or added to in the future. Still, she touted the initial results from the cohort that graduated in 2011-12. “Every college had positive gains, and so that tells us that the career and technical experience that these students had increased their earning potential,” she said.
Harry Painter is a writer for the John W. Pope Center for Higher Education Policy.