Rickety numbers: Volatility of university rankings and policy implications

Michaela Saisana, Béatrice d'Hombres and Andrea Saltelli

Research Policy, 2011, vol. 40, issue 1, 165-177

Abstract: The most popular world university rankings are routinely taken at face value by media and social actors. While these rankings are politically influential, they are sensitive to both the conceptual framework (the set of indicators) and the modelling choices made in their construction (e.g., weighting or type of aggregation). A robustness analysis, based on a multi-modelling approach, aims to test the validity of the inference about the rankings produced in the Academic Ranking of World Universities (ARWU) of Shanghai Jiao Tong University and those produced by the UK's Times Higher Education Supplement (THES). Conclusions are drawn on the reliability of individual university ranks and on relative country or macro regional performance (e.g., Europe versus USA versus China) in terms of the number of top performing institutions. We find that while university and country level statistical inferences are unsound, the inference on macro regions is more robust. The paper also aims to propose an alternative ranking which is more dependant on the framework than on the methodological choices.

Keywords: University; Ranking; Composite; indicator; Uncertainty; analysis; Sensitivity; analysis (search for similar items in EconPapers)
Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (79)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0048-7333(10)00181-2
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:respol:v:40:y:2011:i:1:p:165-177

Access Statistics for this article

Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray

More articles in Research Policy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().