The latest changes in test reporting from the College Board are gathering both approval and protest from education policy analysts and test takers. The College Board has decided to stop “flagging” the results of Scholastic Assessment Tests taken with special accommodations in their reports to colleges.

Disability advocates generally applaud this move, but removing the asterisk from the scores that appear on student transcripts, according to Miriam Freedman in EducationNext, compromises the ability of admissions officers to make informed decisions about college applicants.

The purpose of the SAT has always been twofold. It was designed to assess the “developed verbal and mathematical skills of students,” and to “predict, on average, how students will perform in their first year in college,” Freedman said.

The changes in SAT “flagging” policy took effect in October 2003. Writing in National Review Online, Peter Wood echoed Freedman’s sentiments. He argued in “The SAT Asterisk” that the asterisk is akin to a driver’s license notation that indicates restrictions for corrective lenses. Wood wrote: “I wore glasses when I took the eye exam for my driver’s license. In effect, I took the exam under ‘nonstandard conditions,’ and the Commonwealth of Massachusetts prudently took notice.”

Wood may risk injury and accident if allowed to drive without lenses, but are there really dire consequences if students who receive extra time, or take untimed SAT tests, are made indistinguishable from those who adhere to the strict time limits?

SAT report 2003

Unlike state results from the National Assessment of Educational Progress, the North Carolina SAT Report 2003 makes no mention of the number or percentage of students taking an SAT test with accommodations. Disaggregated SAT statistics report male, female, racial, and the other usual subcategories. No state-level statistics are reported on test-takers with disabilities, or on how many students receive accommodations.

Samuel Abrams, a research fellow in the Center for Basic Research in the Social Sciences at Harvard, reported in “The Demand For Special Accommodations” that since 1987, the total number of students taking the SAT has grown by 18 percent, but the number taking it with accommodations has increased by 300 percent.

“Ordinarily, when the number of disadvantaged students taking the SAT increases, average scores decline,” Abrams writes. Just the opposite has occurred with the SAT. “Verbal scores on nonstandard administrations of the SAT are up by about .2 standard deviation since 1987… But math scores are up by .4 standard deviations, a large gain.”

In North Carolina, average yearly gains in SAT scores since 1989 have been 3.5 points, according the N.C. Department of Public Instruction. Nationally, the average annual gain was 1.3 points over the same period. SAT scores for North Carolina students topped 1000 for the first time ever in 2003.

What does the change in reporting policy mean? Abrams suggested that at least some of the students being granted accommodations were less in need of them than those getting extended or untimed tests back in 1987. Reflecting on the 300 percent increase in special considerations, Abrams asked, “If this has been occurring with flagged scores, what will happen when the scores become unflagged?”

“As of October 2003,” according to Freedman’s ‘Disabling the SAT,’ “admissions officers’ ability to assess the meaning of test results and to make reasonable decisions for all students will be compromised.” Freedman reported that 79 percent of college admissions officers opposed dropping the “nonstandard administration” tag from test scores.

The value of speed

Freedman reported in “Disabling the SAT” that a panel study, conducted by ETS in 1998, found that extra time would not make a significant difference in SAT test scores — 10 points at most on the verbal and 20 points on the math. Students whose overall score in math was below 400 would gain no advantage.

Another study in 1998 allowed disabled students, who had already taken the SAT under standard conditions, to retake the test with extra time. The panel reached the conclusion that the students’ improvement in scores, two to four times larger than those in the earlier 1998 study, were “more representative of their true performance than are the scores they would obtain from a standard administration.” To date, there is no research that compares disabled and nondisabled student results when both get extra time.

Psychometricians on the ETS flagging review panel were asked whether SAT scores achieved with extra time were comparable to those taken in a strict time format. As experts in testing and the comparability of test results, they answered “not sure” and “no” to the questions. As a result, the College Board “ignored industry standards,” and “compromised its flagship product” by removing the test accommodation flags on SAT scores Freedman said.

The potential costs of treating extended-time SAT scores as equivalent to standard-format scores can be significant. If SATs are to remain a reasonable predictor of freshman performance in college, it is important that they do not “overpredict” college achievement. This can mean several things in the college setting, including more “unexpected” failures, pressure to inflate college grades to avoid excessive failure rates, or pressure to make accommodations for these students in their college courses as well.

According to Freedman, the College Board need not have bowed to the “disability” interests. They claimed a violation of the 1973 Rehabilitation Act in their threatened suit against ETS. Citing the 1979 Supreme Court decision in Southeastern Community College v. Davis, Freedman quoted the court as stating, “Section 504 imposes no requirement upon an educational institution to lower or to effect substantial modifications of standards to accommodate a handicapped person.”

“Perhaps the recent attacks on the SAT, especially University of California President Richard Atkinson’s 2001 proposal to eliminate the SAT as a requirement for application, have made the Board wary of controversy,” Freedman’s reports suggested.

In an interesting and somewhat unexpected turn, the New York Times reported that applications for accommodation on the SAT were down 10 percent in 2003 compared to the same July through September period in 2002. The board is turning down more applications than before, and requiring that students have a diagnosis of their disability four months before testing.

Earlier changes in the SAT

The new ruling from ETS isn’t the first change in SAT policy. Significant changes have been under way since 1994. In 1994, the SAT began to emphasize critical reading, use longer reading passages, require math computation, and allow calculators. Antonyms were removed from the test. Psychometric experts “smoothed” the results to make them consistent with earlier versions of the SAT.

Changes in 1995 “recentered” the SAT, re-establishing the average scores for each of the math and verbal portions at 500, and aligning the verbal and math scale scores to make them comparable. This was done to accommodate shifts in the demographics of test-takers. The reference scores for the SAT were established in 1941, and had not been readjusted until the 1995 revision.

The public is now being prepared for the 2005 revision of the SAT. College Board and North Carolina officials have announced that the 2005 test will revise written, math, and verbal test items.

Palasek is policy analyst at the North Carolina Education Alliance and assistant editor at Carolina Journal.