Do Rankings Reflect Research Quality?

Bruno Frey and Katja Rost

CREMA Working Paper Series from Center for Research in Economics, Management and the Arts (CREMA)

Abstract: Publication and citation rankings have become major indicators of the scientific worth of universities and countries, and determine to a large extent the career of individual scholars. We argue that such rankings do not effectively measure research quality, which should be the essence of evaluation. For that reason, an alternative ranking is developed as a quality indicator, based on membership on academic editorial boards of professional journals. It turns out that especially the ranking of individual scholars is far from objective. The results differ markedly, depending on whether research quantity or research quality is considered. Even quantity rankings are not objective; two citation rankings, based on different samples, produce entirely different results. It follows that any career decisions based on rankings are dominated by chance and do not reflect research quality. Instead of propagating a ranking based on board membership as the gold standard, we suggest that committees make use of this quality indicator to find members who, in turn, evaluate the research quality of individual scholars.

Keywords: Rankings; Universities; Scholars; Publications; Citations (search for similar items in EconPapers)
JEL-codes: H43 L15 O38 (search for similar items in EconPapers)
Date: 2008-09
New Economics Papers: this item is included in nep-edu, nep-hpe, nep-ipr, nep-pr~, nep-lab, nep-lam and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (17)

Downloads: (external link)
https://www.crema-research.ch/papers/2008-22.pdf Full Text (application/pdf)
https://www.crema-research.ch/abstracts/2008-22.htm Abstract (text/html)

Related works:
Journal Article: Do rankings reflect research quality? (2010)
Journal Article: Do Rankings Reflect Research Quality? (2010)
Working Paper: Do Rankings Reflect Research Quality? (2008)
Working Paper: Do Rankings Reflect Research Quality? (2008)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cra:wpaper:2008-22

Access Statistics for this paper

More papers in CREMA Working Paper Series from Center for Research in Economics, Management and the Arts (CREMA) Contact information at EDIRC.
Bibliographic data for series maintained by Anna-Lea Werlen ( this e-mail address is bad, please contact ).