Rethinking the Funding Line at the Swiss National Science Foundation: Bayesian Ranking and Lottery

Rachel Heyard, Manuela Ott, Georgia Salanti and Matthias Egger

Statistics and Public Policy, 2022, vol. 9, issue 1, 110-121

Abstract: Funding agencies rely on peer review and expert panels to select the research deserving funding. Peer review has limitations, including bias against risky proposals or interdisciplinary research. The inter-rater reliability between reviewers and panels is low, particularly for proposals near the funding line. Funding agencies are also increasingly acknowledging the role of chance. The Swiss National Science Foundation (SNSF) introduced a lottery for proposals in the middle group of good but not excellent proposals. In this article, we introduce a Bayesian hierarchical model for the evaluation process. To rank the proposals, we estimate their expected ranks (ER), which incorporates both the magnitude and uncertainty of the estimated differences between proposals. A provisional funding line is defined based on ER and budget. The ER and its credible interval are used to identify proposals with similar quality and credible intervals that overlap with the provisional funding line. These proposals are entered into a lottery. We illustrate the approach for two SNSF grant schemes in career and project funding. We argue that the method could reduce bias in the evaluation process. R code, data and other materials for this article are available online.

Date: 2022
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://hdl.handle.net/10.1080/2330443X.2022.2086190 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:usppxx:v:9:y:2022:i:1:p:110-121

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uspp20

DOI: 10.1080/2330443X.2022.2086190

Access Statistics for this article

Statistics and Public Policy is currently edited by Eric Sampson

More articles in Statistics and Public Policy from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().