College Rankings Fail to Measure the Influence of the Institution

M.I.T.’s campus. Its graduates are high earners, but how much credit does M.I.T. deserve for that?

M.I.T.’s campus. Its graduates are high earners, but how much credit does M.I.T. deserve for that?

Students, parents and educators increasingly obsessed with college rankings have a new tool: the Obama administration’s College Scorecard. The new database focuses on a college’s graduation rate, graduates’ median earnings 10 years after graduation and the percentage of students paying back their college loans.

While Scorecard adds potentially valuable information to the dizzying array that is already available, it suffers from many of the same flaws that afflict nearly every other college ranking system: There is no way to know what, if any, impact a particular college has on its graduates’ earnings, or life for that matter.

“It’s a classic example of confusing causation and correlation,” said Frank Bruni, the author of “Where You Go Is Not Who You’ll Be,” a book about the college admissions process, and an op-ed columnist for The New York Times. “Anyone who has taken statistics should know better, but when it comes to colleges, that’s what people do. They throw common sense out the window.”

Of course graduates of the Massachusetts Institute of Technology (average postgraduate earnings $91,600, according to the Scorecard) and Harvard ($87,200) do well. That’s because the students they admit have some of the highest test scores and high school grade point averages in the country, reflecting high intelligence and a strong work ethic — two factors that cause high future earnings. That is generally true regardless of where such students attend college, as long as they go to a reputable four-year institution, various studies have shown.

“It’s absurd,” said Jerry Z. Muller, a professor of history at Catholic University of America and the author of “The Costs of Accountability,” a study of misplaced and misunderstood metrics. “Their graduates have high earnings because they’re incredibly selective about who they let in. And many of them come from privileged backgrounds, which also correlates with high earnings.”

The College Scorecard does not rank colleges, but anyone can use the data to do so. M.I.T. (No. 6 on Scorecard earnings) and Harvard (No. 8) are the only universities in the Scorecard’s top 10 that are also highly ranked by the influential U.S. News and World Report. The other schools have a narrow focus on highly paid skills. The No. 1 school on Scorecard is MCPHS University, whose graduates earn, on average, $116,400. (MCPHS stands for Massachusetts College of Pharmacy and Health Sciences, which is not even ranked by U.S. News.)

But pay, of course, says nothing about the relative quality of different colleges. “If you go to M.I.T. and earn a degree in engineering, you’re going to make more than if you go to Oberlin and major in music performance,” Professor Muller said. “But you already know this. To rank the value of colleges based on the ultimate earnings of their graduates radically narrows the concept of what college is supposed to be for.”

Andrew Delbanco, a professor at Columbia University and author of the book “College: What It Was, Is, and Should Be,” agreed. “Holding colleges accountable for how well they prepare students for postcollege life is a good thing in principle,” he said. “But measuring that preparation in purely monetary terms raises many dangers. Should colleges be encouraged first and foremost to maximize the net worth of their graduates? I don’t think so.”

read more