Field- and time-normalization of data with many zeros: an empirical analysis using citation and Twitter data
Robin Haunschild () and
Lutz Bornmann ()
Additional contact information
Robin Haunschild: Max Planck Institute for Solid State Research
Scientometrics, 2018, vol. 116, issue 2, 997-1012
Abstract Thelwall (J Informetr 11(1):128–151, 2017a. https://doi.org/10.1016/j.joi.2016.12.002 ; Web indicators for research evaluation: a practical guide. Morgan and Claypool, London, 2017b) proposed a new family of field- and time-normalized indicators, which is intended for sparse data. These indicators are based on units of analysis (e.g., institutions) rather than on the paper level. They compare the proportion of mentioned papers (e.g., on Twitter) of a unit with the proportion of mentioned papers in the corresponding fields and publication years. We propose a new indicator (Mantel–Haenszel quotient, MHq) for the indicator family. The MHq is rooted in the Mantel–Haenszel (MH) analysis. This analysis is an established method, which can be used to pool the data from several 2 × 2 cross tables based on different subgroups. We investigate using citations and assessments by peers whether the indicator family can distinguish between quality levels defined by the assessments of peers. Thus, we test the convergent validity. We find that the MHq is able to distinguish between quality levels in most cases while other indicators of the family are not. Since our study approves the MHq as a convergent valid indicator, we apply the MHq to four different Twitter groups as defined by the company Altmetric. Our results show that there is a weak relationship between the Twitter counts of all four Twitter groups and scientific quality, much weaker than between citations and scientific quality. Therefore, our results discourage the use of Twitter counts in research evaluation.
Keywords: Data with many zeros; Citation counts; Altmetrics; Twitter; Mantel–Haenszel quotient (MHq); Equalized mean-based normalized proportion cited (EMNPC); Mean-based normalized proportion cited (MNPC) (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
http://link.springer.com/10.1007/s11192-018-2771-1 Abstract (text/html)
Access to the full text of the articles in this series is restricted.
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:116:y:2018:i:2:d:10.1007_s11192-018-2771-1
Ordering information: This journal article can be ordered from
Access Statistics for this article
Scientometrics is currently edited by Wolfgang Glänzel
More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla ().