Disappointing academic results have led to closer scrutiny of 21st Century Community Learning Centers, the nation’s largest federally funded after-school program. The report, When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program, First Year Findings, 2003, is described as one of the most comprehensive studies of after-school care ever conducted.
The First Year Findings report was prepared by Mathematica Policy Research, Inc. for the U.S. Department of Education. The report gathered data on more than 5,000 elementary- and middle-school students, in 96 after-school programs. The authors say that “[t]he study was designed to examine the characteristics and outcomes of typical programs,” without any attempt to select a “best” program from the group.
The Community Learning Centers were funded by Congress to boost academic achievement. A variety of academic and cultural enrichment activities have been part of the center’s approach. But the Mathematica study shows that the programs don’t generate academic gains. As a result, the Bush administration has decided to request that Congress decrease current funding by 40 percent.
The study used different methods to study middle-school and elementary-school participants, but shows similar results for both groups. In addition to negligible academic improvement, the Mathematica study shows generally low participation rates in the programs. It also shows that higher participation, at least in this first year, did not improve results. According to the 21st Century designers, the programs have been revised to focus more heavily on academics in the future.
Middle-school findings
For the middle-school study, researchers looked at closely matched groups of participating and nonparticipating students. The two groups were drawn from children at the same schools, or at schools in the center’s district.
One question the study explored was how often and how well middle-school students completed homework assignments. As the research discovered, participation in the centers made almost no difference in the rate at which middle schoolers finished assignments. About 83 percent of students in both categories completed homework as assigned. When teachers were surveyed to see whether homework quality was different, they reported slightly more satisfaction with the center students’ work.
Subjective items in the survey asked teachers to assess how hard students appear to have been working. Teachers reported about the same amount of effort for all of the students, and found no differences in attitude toward school work between the groups. Researchers also looked at disciplinary issues, such as skipping school, being sent to the principal’s office for misbehavior, school problems that required a call to parents, or getting detention. Children in the after-school centers had at least as many instances of discipline problems as did nonparticipating children.
The research considered the impact that centers have on students’ test scores. In most subjects, there was no significant difference between the number of students identified by teachers as scoring “above average” or at “very high achievement” levels on tests, regardless of after-school enrichment. An exception appeared in math, however, where scores were slightly higher for center participants. While the math result is considered statistically significant, the effect could be measured in only one-tenth of the sample group. As the study explains, only one-tenth of the students had both before and after math scores available for comparison.
Elementary-school results
After-school care for elementary students is in high demand. There are typically more applicants for Learning Center programs than there are slots for children, so Mathematica was able to study elementary results using an experimental design. Some applicants were randomly assigned to the center programs. Children who were not placed in programs became the matched comparison group. Researchers looked at tests scores, attitudes, and behavior, just as they did with the middle-school group.
The Learning Centers for elementary students were located in 18 elementary schools. The elementary centers emphasized academic achievement over other enrichment areas. A few centers, according to the First Year report, stressed skills needed for state tests or other narrow goals. Others emphasized homework, or reinforced school-day material.
The first annual report described participation in the program as ‘moderate.’ One or two days per week was typical, and attendance declined over the course of the school year. About 25 percent of the students stopped attending altogether after two months.
The children who participated in after-school centers did not show significant academic gains, according to the study. The matched comparison group scored equally well on classroom and standardized tests, and behavior problems occurred as often in one group as in the other.
Family involvement with children
Even though the Community Learning Centers didn’t increase academic scores, participation did have an effect on family involvement. The study found that parents of participating middle schoolers were more likely to participate in school activities, and organizations, than were other parents.
A second finding showed that the number of self-care, or “latchkey” kids was unchanged with the availability of after school programs. Children who were used to spending time alone at home, or in other after school locations, did not switch to centers instead. Kids who did attend programs wound up getting more care from nonparent adults, and less from parents and siblings.
Elementary-school parents didn’t increase their school participation when their children entered programs, according to the report. Middle-school parents, however, volunteered more, attended more school events, and checked for completed homework more often if their child was involved in after-school care.
Behavior
The Mathematica study investigated behavior effects by putting together a composite of student traits. The composite was used to compare students in programs to those who were not enrolled.
Behavior problems occurred just as frequently in after-school students as in students who were not in a program. The composite for elementary students examined goal-setting, cooperation, safety, personal initiative, personal integrity, empathy with others, and feelings of ‘belonging’ with their peers. The composite score was the same for all groups of elementary kids.
Research on middle schoolers’ behavior revealed troubling facts. Participants were more likely to strike teachers or other students, more likely to steal or deliberately destroy property, to lie to their parents, or to sell illegal drugs. Participants were more likely to be victims of other students than were nonparticipants.
New guidelines
In February 2003, the Office of Elementary and Secondary Education issued a set of nonregulatory guidelines designed to clarify current program regulations, and to address concerns about compliance with the No Child Left Behind law. A number of new and more detailed directives will take effect in coming years to try to reform the program.
Private organizations are eligible to receive grants directly for the first time, and program administration will move from the federal to the state level. The guidelines also instruct states to focus their programs on high-poverty, low-performing students. To improve accountability, grant recipients must coordinate programs with local education agencies.
U.S. Undersecretary of Education Eugene Hickock believes the program can be salvaged. “Thanks to this study, we found areas where we can improve,” he told the Dallas Morning News. But others question whether problems can be identified and rectified. Money spent on marginal programs inevitably trades off with money spent elsewhere. In considering a reduction in funding, the administration has signaled a need to look seriously at some of those alternatives.
Palasek is assistant editor of Carolina Journal.