Publication bias is bad for science if not necessarily scientists

Remco Heesen and Liam Kofi Bright

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: It might seem obvious that the scientific process should not be biased. We strive for reliable inference, and systematically skewing the results of inquiry apparently conflicts with this. Publication bias—which involves only publishing certain types of results—seems particularly troubling and has been blamed for the replication crisis. While we ultimately agree, there are considerable nuances to take into account. Using a Bayesian model of scientific reasoning we show that a scientist who is aware of publication bias can (theoretically) interpret the published literature so as to avoid acquiring biased beliefs. Moreover, in some highly specific circumstances she might prefer not to bother with policies designed to mitigate or reduce the presence of publication bias—it would impose a cost in time or effort that she would not see any benefit in paying. However, we also argue that science as a social endeavour is made worse off by publication bias. This is because the social benefits of science are largely secured via go-between agents, various non-experts who nonetheless need to make use of or convey the results of scientific inquiry if its fruits are to be enjoyed by society at large. These are unlikely to be well-informed enough to account for publication bias appropriately. As such, we conclude, the costs of having to implement policies like mandatory pre-registration are worth imposing on scientists, even if they would perhaps not view these costs as worth paying for their own sake. The benefits are reaped by the go-between agents, and we argue that their perspective is quite properly favoured when deciding how to govern scientific institutions.

Keywords: replication crisis; philosophy of statistics; publication bias; preregistration; filedrawer effect; REF fund (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 13 pages
Date: 2025-04-30
New Economics Papers: this item is included in nep-mic and nep-sog
References: Add references at CitEc
Citations:

Published in Royal Society Open Science, 30, April, 2025, 12(4). ISSN: 2054-5703

Downloads: (external link)
http://eprints.lse.ac.uk/127420/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:127420

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().