How accurate are policy document mentions? A first look at the role of altmetrics database

Houqiang Yu (), Xueting Cao, Tingting Xiao and Zhenyi Yang
Additional contact information
Houqiang Yu: Nanjing University of Science and Technology
Xueting Cao: Nanjing University of Science and Technology
Tingting Xiao: Nanjing Library
Zhenyi Yang: Nanjing University of Science and Technology

Scientometrics, 2020, vol. 125, issue 2, No 39, 1517-1540

Abstract: Abstract Policy document mention is considered to indicate the significance and societal impact of scientific product. However, the accuracy of policy document altmetrics data needs to be evaluated to fully understand its strength and limitation. An in-depth coding analysis was conducted on sample policy documents records of Altmetric.com database. The sample consists of 2079 records from all 79 distinct policy document source platforms tracked by the database. Errors about mentioned publications in the policy documents (type A error) are found in 8% of the records, while errors about either the recorded policy documents or the mentioned publications in the altmetrics database (type B error) are found in 70% of the records. In type B error, policy document link error (5% of the records) could be attributable to the policy document website, transcription error (52% of the records) could be attributable to the third-party bibliographic data provider. These two categories of error are relatively minor and may have limited influence on altmetrics research and practices. False positive policy document mention (13% of the records), however, could be attributable to the Altmetric database and may diminish the validity of research based on the policy document altmetrics data. The underlying reasons remain to be further investigated. Considering the high complexity of extracting mentions of publications from various sources and formats of policy documents as well as its short history, Altmetric database has achieved excellent performance.

Keywords: Altmetrics; Data quality; Policy document; Altmetrics database; Accuracy (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-020-03558-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:125:y:2020:i:2:d:10.1007_s11192-020-03558-7

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-020-03558-7

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().