N.C. State University professors Anna Egalite and Stephen Porter find new recipients of Opportunity Scholarships scored significantly higher than their public school counterparts in math, reading, and language arts.
They shared some insight into their study — setting the foundation for future studies into the program — during the Opportunity Scholarship program at the John Locke Foundation’s weekly lecture Monday, June 25.
The study, which recruited 698 students from the public and private schools to take the Iowa Basic Skills Test in spring 2017, looked to find whether the voucher program had academic benefits. Existing voucher recipients, they found, scored significantly higher than their public school counterparts in language arts and higher in math and reading.
“It may be the case that the North Carolina Opportunity Scholarship Program truly has a positive impact on student achievement, perhaps because it reaches highly economically disadvantaged students who have few school choice options in the absence of the program and perhaps the highest potential for academic growth, as a result,” the study reads.
But the researchers cautioned against interpreting the results as wholly representative of all Opportunity Scholarship participants. Researchers were able to recruit nearly 700 private and public students to take the Iowa Basic Skills Test, but fewer than 100 students made it into the final evaluation because researchers couldn’t definitively find a comparable match.
“We can speak with confidence that the students in our sample performed better than statistically similar students in the public schools that are in the same regions of the state,” Egalite said. “But we can’t say anything about the average voucher student is doing in our state.”
In North Carolina, private schools can administer any nationally normed test — such as the Iowa Basic Skills Test or the Stanford Achievement Test. This presents a challenge, because researchers lack a common metric to compare between public school students and Opportunity Scholarship students.
Private school students and Opportunity Scholarship users aren’t required to participate in any evaluation, so researchers had to recruit students and school leaders to take part in the study. The sample, then, could be composed of high academic performers and not representative of the general student population.
Other challenges included a lack of readily available private school academic records and a specific state budget item to evaluate the program.
Despite the limitations, Egalite said, one of the biggest benefits of the study was documenting barriers to accurately evaluate the Opportunity Scholarship program’s academic impact. Doing so allows for future research teams to address those challenges.
“Our legislators clearly have an intention for the program to be studied because they wrote that into the statute, so I think it’s very important for us to demonstrate that the program is not set up to be evaluated in a gold standard way right now,” Egalite said.
One way to achieve a more accurate study entails capping the program and delaying some students access to the Opportunity Scholarship to create a control group. Researchers could then compare students who got into the program with those who didn’t using the same standardized test.
Another way involves requiring families to participate in a program evaluation upon accepted. Either way, Egalite said, more research is needed into the Opportunity Scholarship program before the case can be settled.