Screening p-hackers: Dissemination noise as bait

Federico Echenique and Kevin He

Department of Economics, Working Paper Series from Department of Economics, Institute for Business and Economic Research, UC Berkeley

Abstract: We show that adding noise before publishing data effectively screens [Formula: see text]-hacked findings: spurious explanations produced by fitting many statistical models (data mining). Noise creates baits that affect two types of researchers differently. Uninformed [Formula: see text]-hackers, who are fully ignorant of the true mechanism and engage in data mining, often fall for baits. Informed researchers, who start with an ex ante hypothesis, are minimally affected. We show that as the number of observations grows large, dissemination noise asymptotically achieves optimal screening. In a tractable special case where the informed researchers theory can identify the true causal mechanism with very few data, we characterize the optimal level of dissemination noise and highlight the relevant trade-offs. Dissemination noise is a tool that statistical agencies currently use to protect privacy. We argue this existing practice can be repurposed to screen [Formula: see text]-hackers and thus improve research credibility.

Keywords: dissemination noise; p-hacking; privacy; research integrity (search for similar items in EconPapers)
Date: 2024-05-21
New Economics Papers: this item is included in nep-inv, nep-mac and nep-sog
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.escholarship.org/uc/item/6sm4w1jf.pdf;origin=repeccitec (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cdl:econwp:qt6sm4w1jf

Access Statistics for this paper

More papers in Department of Economics, Working Paper Series from Department of Economics, Institute for Business and Economic Research, UC Berkeley Contact information at EDIRC.
Bibliographic data for series maintained by Lisa Schiff ().