Sceptics and champions: participant insights on the use of partial randomization to allocate research culture funding

Catherine Davies and Holly Ingram

Research Evaluation, 2025, vol. 34, 154-78

Abstract: As part of the shift towards a more equitable research culture, funders are reconsidering traditional approaches to peer review. In doing so, they seek to minimize bias towards certain research ideas and researcher profiles, to ensure greater inclusion of disadvantaged groups, to improve review quality, to reduce burden, and to enable more transformative research. This paper presents the rationale and findings from a trial of partially randomized funding allocation (PRA) used to internally distribute Research England’s Enhancing Research Culture fund at the University of Leeds. Of 26 eligible applications, six fell in the upper midfield on quality assessment and were randomized. Of this subset, one received the available funding. Qualitative data from applicants, reviewers, and moderators in the trial suggest modest gains regarding the reduction or perception of bias and efficiency of peer review. There were variable benefits of the feedback that PRA provides. A range of concerns emerged about PRA’s perceived fairness, bluntness, and threat to quality. Based on the behavioural responses we captured, we present strategic and operational recommendations for professionals working in research evaluation who are interested in adopting PRA as a way of improving research culture. For example, we highlight the importance of (1) sharing data on the role of chance in traditional peer review, and (2) considering the benefits and risks of PRA at both group and individual applicant levels.

Keywords: grant peer review; research funding allocation; partial randomization; modified lottery; research culture (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvaf006 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:34:y:2025:i::p:154-78.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().