The language of peer review reports on articles published in the BMJ, 2014–2017: an observational study

Alberto Falk Delgado (), Gregory Garretson and Anna Falk Delgado
Additional contact information
Alberto Falk Delgado: Uppsala University
Gregory Garretson: Uppsala University
Anna Falk Delgado: Karolinska Institute

Scientometrics, 2019, vol. 120, issue 3, No 13, 1225-1235

Abstract: Abstract To analyse the words and expressions used in peer reviews of manuscripts that were later published as original research in the BMJ. Secondary aims were to estimate the differences in net sentiment between peer review reports on manuscripts subject to one or more rounds of peer review and and review reports on initially rejected manuscripts that were accepted after appeal. This observational study included all peer review reports published in the BMJ from September 2014 until the end of 2017. The study analysed the frequency of specific words in peer review reports for accepted manuscripts, identifying the most commonly occurring positive and negative words and their context, as well as the most common expressions. It also quantified differences in net sentiment in peer review reports between manuscripts accepted after appeal and manuscript accepted without appeal. The dataset consisting of 1716 peer review reports contained 908,932 word tokens. Among the most frequent positive words were “well”, “important”, “clear”, “while the negative words included “risk”, “bias”, and “confounding”. The areas where the reviewer makes the most positive and negative comments included: “well-written paper”, “well-written manuscript”, “this is an important topic”, “answers an important question”, “high risk of bias” and “selection bias”. The sentiment analysis revealed that manuscripts accepted after appeal had lower scores on review reports for joy and positive sentiment, in addition to having higher scores for negative words expressing sadness, fear, disgust and anger compared with manuscripts that were not initially rejected. Peer review comments were mainly related to methodology rather than the actual results. Peer review reports on initially rejected manuscripts were more negative and more often included expressions related to a high risk of bias.

Keywords: BMJ; Peer review; Reviewer comments; Linguistics; Medicine; Open peer review (search for similar items in EconPapers)
Date: 2019
References: View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-019-03160-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:120:y:2019:i:3:d:10.1007_s11192-019-03160-6

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-019-03160-6

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().