Methods to account for citation inflation in research evaluation

Alexander M. Petersen, Raj K. Pan, Fabio Pammolli () and Santo Fortunato

Research Policy, 2019, vol. 48, issue 7, 1855-1865

Abstract: Quantitative research evaluation requires measures that are transparent, relatively simple, and free of disciplinary and temporal bias. We document and provide a solution to a hitherto unaddressed temporal bias – citation inflation – which arises from the basic fact that scientific publication is steadily growing at roughly 4% per year. Moreover, because the total production of citations grows by a factor of 2 every 12 years, this means that the real value of a citation depends on when it was produced. Consequently, failing to convert nominal citation values into real citation values produces significant mis-measurement of scientific impact. To address this problem, we develop a citation deflator method, outline the steps to generalize and implement it using the Web of Science portal, and analyze a large set of researchers from biology and physics to demonstrate how two common evaluation metrics – total citations and h-index – can differ by a remarkable amount depending on whether the underlying citation counts are deflated or not. In particular, our results show that the scientific impact of prior generations is likely to be significantly underestimated when citations are not deflated, often by 100% or more of the nominal value. Thus, our study points to the need for a systemic overhaul of the counting methods used evaluating citation impact – especially in the case of researchers, journals, and institutions – which can span several decades and thus several doubling periods.

Keywords: Measurement error; Career evaluation; Citation analysis; Science policy (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (20)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0048733319301003
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:respol:v:48:y:2019:i:7:p:1855-1865

DOI: 10.1016/j.respol.2019.04.009

Access Statistics for this article

Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray

More articles in Research Policy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().