Publications & Products
Beat Them, Join Them, or Leave Them Be?

By Kevin Carey




Rise and Shine?
The ups and downs of the rankings game

Every year, it happens: The U.S. News & World Report "Best Colleges" issue hits newsstands, and the outcry from campus presidents, administrators, and higher education observers begins. The love-hate relationship with the rankings plays out publicly, with campuses that fared well touting the good news wherever possible and campuses that perhaps didn't fare so well struggling with how to communicate, if at all, about their ranking.

It's a cyclical phenomenon-the uproar begins; people voice the same complaints, argue the pros and cons, ask the same questions, find no solid answers or solutions; and then it dies down. Around and around the U.S. News conversation goes, year after year. Surely there are other publications that compare and even rank institutions, but it's U.S. News that gets the most attention, due in part to its longevity.

Parents and students want and need a starting point, a basis for comparison, to help them wade through information about the more than 3,000 colleges and universities in the United States. But if that's all the U.S. News rankings provided, perhaps there wouldn't be so much controversy. Frankly, most people just want to know who's best. And that's where the issue becomes a little more convoluted. What does it mean to be the best? Who should decide who's best? What is it that people really want to know? What is it that institutions should be measuring, assessing, and providing?

The theme of this issue of CURRENTS explores the idea that there's a crisis in public discourse-the public has no appetite for nuance, people only seem to want the highlights, the bullet points, the sound bites on a given topic, especially one that might seem too big and complicated to dissect. The rankings debate, in more ways than one, brilliantly illustrates this point.

CURRENTS has covered the rankings ruckus in the past, but this time we revisit the issue with a specific eye on illuminating the complex conversation with thoughtful perspectives from Colin Diver, president of Reed College, and Kevin Carey, a researcher at Education Sector.

 

Earlier this year, presidents from 12 private institutions launched a new salvo in the ongoing debate about the U.S. News & World Report college rankings. In a widely circulated letter sponsored by the nonprofit Education Conservancy and covered in Time magazine, the Washington Post op-ed page, and other public forums, the presidents called for a boycott of the controversial, hugely popular list of "America's Best Colleges."

The idea has intuitive appeal-if colleges and universities can act as one, they can reclaim control over their public images and halt the growing influence of U.S. News. The flaws and pernicious effects of the magazine's rankings are well-known. Ninety-five percent of the rankings are based directly or indirectly on just three factors: wealth, fame, and exclusivity. As a result, institutions are focusing inordinate attention on fundraising, marketing, and attracting faculty with outsized scholarly reputations-at the expense of the core missions of access and undergraduate learning. The president of Arizona State University even has performance bonuses tied to the U.S. News rankings written into his contract.

But as attractive as an anti-rankings revolt may be, it is unlikely to succeed. U.S. News is a private company, immune from government regulation. The maga­zine has clearly stated that it will rank colleges whether or not they submit data, fill out the magazine's reputational survey, or otherwise participate in the rankings process. Most of the data that form the rankings, like admissions rates, graduation rates, spending levels, and SAT scores, can be found in publicly available government databases. Institutions can ignore rankings, but they will never be able to prevent them from existing.

And there will always be wealthy, famous, exclusive institutions that see the rankings as in their best interests-like Yale University, whose president, Richard Levin, recently said in the Yale Daily News: "If the [boycott] letter says abolish any attempt by the press to characterize strong versus weak colleges, I would be opposed to that. Schools have to be accountable, and it's part of our tradition of free press to have external evaluators of the performance of our schools."

Colleges and universities can't beat the rankings. Therefore, they have two options: join them, or accept them and leave them be.

Risky business

Accepting them carries significant risk. Twenty-first century students see themselves as information-empowered consumers with a broad array of choices in the market. Higher education has historically failed to meet the need for adequate consumer information, instead relying on slick brochures that seem to offer little more than pictures of brightly lit science labs and leaf-strewn lawns. U.S. News simply stepped in to fill that void, and its central position there has never been seriously challenged.

As a result, the higher education community has essentially allowed a newsmagazine to define the terms of success in higher learning. Even as recommendations for increased accountability from Secretary of Education Margaret Spellings' Commission on the Future of Higher Education are hotly debated, a for-profit magazine continues to create what amounts to a national higher education accountability system, complete with widely publicized measurements of "quality" (the rankings) and strong incentives for colleges and universities to improve their standing. As long as this remains the case, the pressure on institutional leaders to boost their rankings will only grow stronger. Students, along with the best ideals of higher education, will lose out.

Reform school

That leaves the option of joining the rankings-not by acceding to the U.S. News definition of higher education quality, but by replacing it with something better. Until recently, this would have been impossible. There simply wasn't enough good, comparable information available to construct an alternate rankings system. But that data deficit is rapidly being filled. In just the past five years, whole new classes of information have become available. In combination, they could be used to create a newer, better rankings regime.

The National Survey of Student Engage­ment, for example, is now being used by hundreds of colleges and universities to measure effective educational practices such as active and collaborative learning and student-faculty interaction. The Collegiate Learning Assessment, developed by a former subsidiary of the RAND Corporation, is providing new measures of the "value-added" components that colleges contribute to cross-disciplinary skills like communications, analytic reasoning, and critical thinking. The U.S. News rankings are based on SAT scores, which tell us where students were a year before they arrived at college. The CLA tells us how much students improve while they're at college.

States such as Florida have developed sophisticated statewide data systems that allow officials to calculate the educational and labor market outcomes of college graduates years after they finish school. These data systems also allow for graduation rate calculations that-unlike the current federal measures-take into account students who transfer from one institution to another. Meanwhile, accreditors in disciplines such as engineering and teaching are creating new processes to gauge the quality of academic programs, based not on input measures like credentials and curricula but on the success of program graduates in their chosen field. New alumni survey instruments, like the Collegiate Results Survey piloted in Pennsylvania, have been developed to gather information about institutions' contribution to nonvocational, nonacademic areas such as civic engagement, physical fitness, spiritual lives, and commitment to the arts.

In combination, these measures could be used to form new college rankings far superior to those sold by U.S. News. They wouldn't be perfect, of course-no rankings list can be. When complex institutions are boiled down to a single number, much can be lost in translation. That said, students can only choose one college, so in a sense the reductive rankings process is an unavoidable element of a consumer-driven market. That's why rankings are also unavoidable. We can't choose whether to have rankings or not, only whether they'll be good or bad.

Unlocking the vault

There's a catch, of course: Most of these data aren't publicly available for all institutions, either because state data systems vary in quality, or because institutions choose to keep it secret. Most NSSE results, for example, can't be used by students choosing where to go to college because institutions don't release them to the public. In defending its rankings, U.S. News correctly notes that there are no good measures that it could be using for its rankings but is choosing not to. Indeed, the most complete repository of voluntarily disclosed NSSE data can be found on the U.S. News Web site itself. To dislodge U.S. News from the minds of the public, higher education will have to be far more transparent and forthcoming about its successes and failures than it has historically chosen to be.

Some of the reluctance to release the new data is understandable-the U.S. News rankings are bad for most colleges and universities, but not all. Those that have long stood atop the higher education status hierarchy have nothing to gain from changing a system that reinforces their primacy in a very public way.

But for the vast majority of colleges and universities, a new rankings system-scary though it might be-would be a boon. Of all the flaws of the current rankings system, none is worse than this: It gives most institutions no chance to be recognized for being good-or even great-at what they were meant to be. Public universities, regional campuses, minority-serving institutions, colleges with a mission to serve first-generation students-these are the higher education institutions that actually educate the vast majority of American college students. Yet U.S. News affords them no opportunity to be fully rewarded for excellence, no recognition for serving at-risk students whose educational lives count just as much as the privileged handful enrolled at the few elite institutions perennially competing to be America's "best" college. Creating a level playing field for all institutions to compete, reorienting the public perception of higher education quality toward the core values of scholarship, education, and learning-these are goals to organize around, in letters and much more. And unlike the quixotic quest to banish college rankings from the public mind, this is a fight higher education could actually win.

About the Author Kevin Carey

Kevin Carey is policy and research manager at Education Sector, a think tank based in Washington, DC.

 

Comments

 

Add a Comment

You must be logged in to comment . Your name and institution will show with your comment.