Development and application of a comprehensive glossary for the identification of statistical and methodological concepts in peer review reports

Ivan Buljan, Daniel Garcia-Costa, Francisco Grimaldo, Richard A. Klein, Marjan Bakker and Ana Marušić

Journal of Informetrics, 2024, vol. 18, issue 3

Abstract: The assessment of problems identified by peer researchers during peer review is difficult because the content of these reports is typically confidential. The current study sought to construct and apply a glossary for the identification of methodological and statistical concepts mentioned in peer review reports. Three assessors created a list of 1,036 different terms in 19 categories. The glossary was tested on the confidential PEERE database, a sample of 496,928 peer review reports from various scientific disciplines. The most frequently mentioned terms were related to data presentation (found in 40.3 % of the reports) and parametric descriptive statistics (33.3 %). Review reports suggesting a rejection were more likely to mention methodological issues, whereas statistical issues were raised more frequently in review reports recommending revisions. Across disciplines, methodological issues were more frequently mentioned in social sciences (64.1 %), while health and medical sciences were more predictive for the identification of statistical issues (40.1 %). Female reviewers identified more statistical issues compared to male reviewers. These results indicate that the glossary could be used as an additional tool for the assessment of the content of peer review reports and for understanding what help authors may need in writing research articles.

Keywords: Peer review; Dictionary-based approach; Statistical glossary; Discipline differences; Peere database; Research quality assessment (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1751157724000683
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:infome:v:18:y:2024:i:3:s1751157724000683

DOI: 10.1016/j.joi.2024.101555

Access Statistics for this article

Journal of Informetrics is currently edited by Leo Egghe

More articles in Journal of Informetrics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().