Scientific publishing without gatekeeping: an empirical investigation of eLife’s new peer review process

Rüdiger Mutz (), Lutz Bornmann and Hans-Dieter Daniel
Additional contact information
Rüdiger Mutz: University of Zurich
Lutz Bornmann: Administrative Headquarters of the Max Planck Society
Hans-Dieter Daniel: University of Zurich

Scientometrics, 2025, vol. 130, issue 9, No 12, 5067-5092

Abstract: Abstract At the end of January 2023, eLife introduced a new publishing model (alongside the old-traditional-publishing model): all manuscripts submitted as preprints are peer-reviewed and published if they are deemed worthy of review by the editorial team (“editorial triage”). The model abandons the gatekeeping function and retains the previous “consultative approach to peer review”. Even under the changed conditions, the question of the quality of judgements in the peer review process remains. In this study, the reviewers’ ratings of manuscripts submitted to eLife were examined in terms of both descriptive comparisons of peer review models, and the following selected quality criteria of peer review: interrater agreement and interrater reliability. eLife provided us with the data on all manuscripts submitted in 2023 according to the new publishing model (group 3, N = 3,846), as well as manuscripts submitted according to the old publishing model (group 1: N = 6,592 submissions from 2019; group 2: N = 364 submissions from 2023). The interrater agreement and interrater reliability for the criteria “significance of findings” and “strength of support” were similarly low, as previous empirical studies for gatekeeping journals have shown. The fairness of peer review is not or only slightly compromised. We used the empirical results of our study to recommend several improvements to the new publishing model introduced by eLife as for example, increasing transparency, masking author identity or increasing the number of expert reviewers.

Keywords: Journal peer review; Interrater agreement; Interrater reliability; Gatekeeping; Open science (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-025-05422-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:130:y:2025:i:9:d:10.1007_s11192-025-05422-y

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-025-05422-y

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().