Choose Your Moments: Peer Review and Scientific Risk Taking

Richard Carson, Joshua Graff Zivin and Jeffrey Shrader

No 31409, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Science funding agencies such as the NIH, NSF, and their counterparts around the world are often criticized for being too conservative, funding incremental innovations over more radical but riskier projects. One explanation for their conservatism is the way the agencies use peer review of scientific proposals. Peer review is the cornerstone of research allocation decisions, but agencies typically base decisions on a simple average of peer review scores. More novel ideas are less likely to gain consistently high ratings across evaluators and are less likely to be funded. Using a discrete choice experiment conducted with a large sample of active biomedical researchers, we find that—in contrast to funding agencies—scientists systematically prefer to fund projects with more reviewer dissensus. Rather than purely focusing on the first moment of the distribution of reviewer scores, they also value the second moment. Further, scientists with the greatest domain expertise on a proposal are more enthusiastic about dissensus, and while appetite for dissensus shrinks as budgets become tighter, it does not disappear completely. Applying our estimates to prior studies mimicking NIH’s review process shows that incorporating expert scientists’ preferences for dissensus would change marginal funding decisions for ten percent of projects worth billions of dollars per year.

JEL-codes: H40 O3 O38 (search for similar items in EconPapers)
Date: 2023-06
New Economics Papers: this item is included in nep-dcm, nep-ppm and nep-sog
Note: EH PE PR
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.nber.org/papers/w31409.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:31409

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w31409

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().