How similar are field-normalized citation impact scores obtained from OpenAlex and three popular commercial databases? An empirical comparison based on large German universities

Thomas Scheidsteger (), Robin Haunschild () and Lutz Bornmann ()
Additional contact information
Thomas Scheidsteger: IVS-CPT, Max Planck Institute for Solid State Research
Robin Haunschild: IVS-CPT, Max Planck Institute for Solid State Research
Lutz Bornmann: IVS-CPT, Max Planck Institute for Solid State Research

Scientometrics, 2025, vol. 130, issue 7, No 10, 3537-3569

Abstract: Abstract OpenAlex is a freely available bibliographic database that can be used for bibliometric studies. In this study, we compared certain field-normalized citation scores (NCS) from OpenAlex with those from three commercial databases (Web of Science, Scopus, and Dimensions). We were interested in the question whether the NCS from OpenAlex are comparable to those from the commercial databases and can be alternatively used in evaluative bibliometrics. The NCS have been calculated for nearly 335,000 papers published by 48 German universities in four main subject areas between 2013 and 2017. We found varying but overall strong agreement between the scores according to Lin’s concordance correlation coefficient. Separating the publication set along the single universities and moreover along the four main subject areas involved revealed significant differences at the level of single papers but also gave indications on how to possibly mitigate outlier cases. We calculated mean normalized citation scores for the 48 universities and found that the agreements across different databases are low. On the one hand, the results suggest that comparisons of universities using NCS across different databases should be avoided. On the other hand, the difference of the concordance correlation coefficients at paper and university level is a good example for the problem of ecological fallacy in bibliometrics: The mean impact is not representative for the single papers’ impact in the set.

Keywords: Citation impact; MNCS; OpenAlex; Web of science; Scopus; Dimensions; German universities (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-025-05338-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:130:y:2025:i:7:d:10.1007_s11192-025-05338-7

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-025-05338-7

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().