Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution

Charles W. Fox ()
Additional contact information
Charles W. Fox: University of Kentucky

Scientometrics, 2017, vol. 113, issue 1, No 23, 465-477

Abstract: Abstract Journal peer review relies on the willingness of researchers to volunteer their time to review manuscripts. However, editors often have difficulty recruiting reviewers, and this difficulty can vary quite substantially among manuscripts. This study examines whether the difficulty recruiting reviewers influences outcomes of the peer review process at six journals of ecology and evolution. The difficulty editors had recruiting reviewers varied substantially among papers, with editors successfully recruiting the first two people invited just 22% of the time, and being declined by two or more invitees for more than half (56%) of reviewed papers. Papers for which editors had more difficulty recruiting reviewers were more likely to be declined at all six journals, with an increase in the odds of acceptance ranging from a low of 3.5 ± 1.2% to a high of 17.3 ± 2.0% for each 10% increase in the proportion of reviewers agreeing to review. Papers for which editors had more difficulty recruiting reviewers were also reviewed less positively at all six journals, and this influence on review scores explained most but not all of the influence of recruitment difficulty on outcomes. Reviewers invited close together in sequence (without many declined invitations between them) were more consistent in the scores they submit than were reviewers invited more greatly separated in sequence, suggesting that editors recruit different kinds of reviewers early versus late in the reviewer invitation sequence. However, the scores submitted by later-recruited reviewers were not less predictive of the editor’s decision than were scores of early-recruited reviewers. The influence of reviewer recruitment difficulty on decisions, although of small effect, should be considered among the diversity of variables that influence outcomes of the editorial and peer review process at academic journals.

Keywords: Bias in peer review; Inter-rater reliability; Peer review; Referees (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-017-2489-5 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2489-5

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-017-2489-5

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().