Idea of the day: ditch standardized tests

examsWhy do colleges and universities require millions of students each year to take standardized tests as part of their college applications?  The conventional rationale goes something like this: there is a vast quality difference in the nation’s high schools, so GPA alone may not accurately reflect the true ability of the students.  After all, an A in English at Phillips Exeter is not the same as an A at East Memphis Public High, but a 2400 on the SATs is the same everywhere.  Standardized tests therefore put students on the same evaluative plane.  The test scores are more useful for gauging the student’s academic aptitude than GPAs.

Sounds good, except that it’s not really true.  Critics have long charged that flaws and biases in these tests make them bad predictors of college success.  The latest and important evidence comes from the new book Crossing the Finishing Line. In a brief review of the book, Chad Alderman notes that the data in the book shows that standardized test scores have little predictive power of student graduation rates in college.  High school GPAs are “three to five times more important” in predicting whether a student will graduate from college than SAT or ACT scores.  Moreover, when high school quality is accounted for, the predictive power of SAT and ACT entirely disappears and even becomes negative.

What should we make of this data?  If the data shows what it purports to show, then a major — perhaps the only — rationale for requiring and using standardized test scores in college admissions has just been dealt a fatal blow.

And on the other side, there is a host of possible negative effects of using standardized tests.  The critics of standardized tests have accused it of being biased against low-income students in favor of more affluent students, of spawning a 310-million-dollar-per-year test-prep industry, of encouraging rote preparations in schools, of harboring racial and gender biases.  The sad truth is that even the supporters of standardized tests may only favor the tests in spite of all their flaws because “this is the only thing we have.

Is it really the only thing we have?  If we want to predict student performance and admit only those who are likely to succeed in college, colleges could simply look at objective measures of high school quality and use that in conjunction with student grades to determine how much that GPA should be weighted compared to other factors and candidates.

And if we wish to reward the students who succeeded in spite of adversity, we should use the same objective measure of high school quality and give students who did well in the worst schools extra points in their applications.

This brings me to another concern I have with standardized test scores.  The scores, as meaningless and unpredictive as they might be, give ammunition to those who accuse affirmative action programs of being unfair to meritorious applicants who lose out to “less qualified” minorities.  The scores are often presented as a neutral and quantifiable indicator of how much less qualified the minority candidates are.

The plaintiff in Gratz v. Bollinger, for example, attacked the University of Michigan for giving race 20 points on a 100-point scale while giving only 12 points for having a perfect SAT score.  And as I have written here previously, a Princeton researcher found that being African-American is equivalent to having 230 extra points in a 1600-point SAT exam, and being Hispanic is equivalent to 185 points.

How closely a school tracks admission to standardized testing scores has become a proxy for whether the school is “meritocratic.”  Disparities in testing scores among different ethnic groups is a favorite examples that opponents of affirmative action use to criticize it.  But their criticism has a lot less force if the testing scores themselves are poor proxies for student ability.

A perfect score on the SAT is indeed the same everywhere.  But it turns out that the perfect score doesn’t tell us very much at all about what we want to know.


Too focused on the racial achievement gap?

thegapIt seems that our society has an obsession for anything related to race. Last Wednesday, the National Assessment of Educational Progress (NAEP) released disappointing results for math exams that the nation’s fourth and eighth graders took earlier this year. The results were worrisome, to say the least: only 39% of fourth graders and 34% of eighth graders performed above the proficiency level. In addition to the predictable stories about low scores and the lack of improvements, many media outlets also reported on the achievement gaps among the nation’s various ethnic and racial groups.

The gaps are huge, and they have widened slightly from two years ago. The New York Times reports that on the 500-points exam where the average score for eighth graders was 282, the average white students scored 293 points, black students averaged only 261 points while hispanics scored 266 points. The 32-point gap between white and black students represents about 3 years worth of learning.

According to National Center for Education Statistics, while 82% of white students have “basic” math skills and 43% have “proficient” skills, only 49% of black students have basic skills, and a dismal 12% are proficient. Hispanic students once again fare slightly better than black students (56%, 17%) but are still leagues behind white students.

These numbers are certainly cause for concern, and there is no doubt that our schools are failing a large portion of our nation’s children, but why do we focus on race? A short foray into the nifty and slightly addictive NAEP website (where you can build your own charts and compare the performance of different groups based on hundreds of different criteria) reveal a host of even more dramatic achievement gaps.

One such gap is the rich-poor gap.  While direct economic data for the students are not available from the NAEP exams, a useful proxy is whether the student is eligible for the federal free lunch program. Here, the gap is 27 points, almost as much as the black-white achievement gap and exactly the same as the white-hispanic achievement gap. Only 17% of those eligible for free school lunches have reached a proficient level, compared to 45% of those not eligible.

And what about the rich-school-district-poor-school-district gap? Here, the difference is even more dramatic.  The nation’s richest districts — those with 0% of their students eligible for free lunches — have an average score of 302 — 20 points above the national average. The poorest districts — those with 100% of eligible students, averaged a score of 258. The 44 points gap represents about four years of education.

And what about the state achievement gaps? Massachusetts eighth graders had the highest average scores in the nation with 299 points. Minnesota came in second at 294, and a slew of other states, all of them northern, follow them at 293. At the bottom, DC schools are the worst in the nation, clocking in at 254 points, 45 points behind Massachusetts. Mississippi and Alabama are at 265 and 269 points each.

I can come up with a host of other interesting achievement gaps: students whose parents graduated from college vs. students whose parents did not finish high school (30 points), students with fewer than 10 books in the home and students with 100 books or more (41 points), and a public-private school gap (14 points).

All of these achievement gaps have something to do with the socio-economic status of the students. It won’t surprise anyone to learn that students from more affluent backgrounds, whose parents are better educated and whose neighborhoods are safer, learn more in school and perform better at standardized exams. It also won’t surprise anyone that the schools in richer neighborhoods have more resources and are better run, thereby making teaching and learning more effective. The differences that these scores have shown are dramatic and infuriating.

Which is why I find it even harder to understand our fixation on race. Is it really so surprising, given that a larger percentage of this nation’s black and hispanic citizens live in poverty and have a lower average income, that their children would perform worse in school? It seems that our concern with the racial achievement gap is, in large part, a concern with the underlying socioeconomic ills that plague a disproportionate percentage of minorities in our nation. Rather than clarifying the problems that these youngsters and our schools face, our fixation on race obfuscates it.