Analyzing the impact of reputational bias on global university rankings based on objective research performance data: the case of the Shanghai Ranking (ARWU)

Vicente Safón () and Domingo Docampo ()
Additional contact information
Vicente Safón: University of Valencia
Domingo Docampo: University of Vigo

Scientometrics, 2020, vol. 125, issue 3, No 15, 2199-2227

Abstract: Abstract This paper analyzes the potential impact of reputational bias on academic classifications based upon current research performance measures. Empirical evidence supports the existence of a noticeable effect of reputational bias in the Academic Ranking of World Universities (ARWU), informally known as the Shanghai Ranking. Despite using reliable, objective data, the results of the ranking are partially contaminated by the halo effect that pervades peer review processes and citation practices behind two of ARWU’s main indicators: highly cited researchers (HiCi), and papers published in Nature and Science (N&S). ARWU results may contribute to reinforcing reputational bias through a vicious feedback loop. The study describes a method for quantifying the bias present in the aforementioned indicators. Our findings reveal that bias exists in N&S but not in HiCi, and that while it benefits top universities and Japanese universities, it stifles Australian counterparts. The paper paves the way for a new, interesting debate that will require future research to identify and cope with shortcomings derived from the impact of reputational bias in journal procedures both when evaluating and accepting manuscripts, as well as in the methodology of global university rankings.

Keywords: ARWU; Ranking; University; Reputation; Regression; 62J05; 62P25 (search for similar items in EconPapers)
JEL-codes: I23 L14 (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-020-03722-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:125:y:2020:i:3:d:10.1007_s11192-020-03722-z

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-020-03722-z

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().