Can peer review accolade awards motivate reviewers? A large-scale quasi-natural experiment

Houqiang Yu (), Yian Liang and Yinghua Xie
Additional contact information
Houqiang Yu: Sun Yat-sen University
Yian Liang: Sun Yat-sen University
Yinghua Xie: Sun Yat-sen University

Palgrave Communications, 2024, vol. 11, issue 1, 1-13

Abstract: Abstract The utilization of accolade awards to motivate reviewers is widespread, but their effectiveness remains uncertain. We aim to explore how receiving an accolade award affects reviewers’ subsequent number of reviews. In contrast to small-scale, unrepresentative experiments, we perform a large-scale, global, and all-disciplinary analysis based on quasi-natural experiments. By integrating the Publons, ORCID, MAG, SciSciNet, and OpenAlex databases, a large dataset that tracks the reviewers’ annual number of reviews and their bibliometric indicators is compiled, encompassing 179,794 individuals. Among them, 6605 individuals who receive the “Publons Global Peer Review Award” in 2018 are constituted as the experimental group. Those non-winners are then matched to form a control group. Using propensity score matching (PSM), 711 and 762 reviewers are matched as the experimental and control groups respectively. An analysis employing the difference-in-differences (DiD) method is conducted to examine the impact of an accolade award on reviewers’ subsequent number of reviews. It is found that, following the receipt of an accolade award, reviewers, on average, reviews about four fewer manuscripts, with the reduction exhibiting a V-shaped pattern. Additional analyses are conducted to examine how individual differences and socio-economic factors influence the awarding effects. Besides, we analyze the mechanisms underlying the awarding effect and propose strategies to motivate reviewers. In brief, the sharp decrease in marginal utility of accolades, the voluntary nature of peer review, and the unexpected properties of accolade awards are the potential mechanisms that generate the negative effect. The academic community should reassess the existing incentive strategies for reviewers.

Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1057/s41599-024-04088-w Abstract (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-04088-w

Ordering information: This journal article can be ordered from
https://www.nature.com/palcomms/about

DOI: 10.1057/s41599-024-04088-w

Access Statistics for this article

More articles in Palgrave Communications from Palgrave Macmillan
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().