Relevance assessments, bibliometrics, and altmetrics: a quantitative study on PubMed and arXiv

Timo Breuer (), Philipp Schaer () and Dirk Tunger ()
Additional contact information
Timo Breuer: TH Köln - University of Applied Sciences
Philipp Schaer: TH Köln - University of Applied Sciences
Dirk Tunger: TH Köln - University of Applied Sciences

Scientometrics, 2022, vol. 127, issue 5, No 14, 2455-2478

Abstract: Abstract Relevance is a key element for analyzing bibliometrics and information retrieval (IR). In both domains, relevance decisions are discussed theoretically and sometimes evaluated in empirical studies. IR research is often based on test collections for which explicit relevance judgments are made, while bibliometrics is based on implicit relevance signals like citations or other non-traditional quantifiers like altmetrics. While both types of relevance decisions share common concepts, it has not been empirically investigated how they relate to each other on a larger scale. In this work, we compile a new dataset that aligns IR relevance judgments with traditional bibliometric relevance signals (and altmetrics) for life sciences and physics publications. The dataset covers PubMed and arXiv articles, for which relevance judgments are taken from TREC Precision Medicine and iSearch, respectively. It is augmented with bibliometric data from the Web of Science and Altmetrics. Based on the reviewed literature, we outline a mental framework supporting the answers to our research questions. Our empirical analysis shows that bibliometric (implicit) and IR (explicit) relevance signals are correlated. Likewise, there is a high correlation between biblio- and altmetrics, especially for documents with explicit positive relevance judgments. Furthermore, our cross-domain analysis demonstrates the presence of these relations in both research fields.

Keywords: Relevance assessments; Polyrepresentation; Altmetrics; Relevance theory; Information retrieval; Test collections (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-022-04319-4 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:127:y:2022:i:5:d:10.1007_s11192-022-04319-4

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-022-04319-4

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().