Detecting p‐Hacking

Graham Elliott (), Nikolay Kudrin and Kaspar Wüthrich

University of California at San Diego, Economics Working Paper Series from Department of Economics, UC San Diego

Abstract: We theoretically analyze the problem of testing for p‐hacking based on distributions of p‐values across multiple studies. We provide general results for when such distributions have testable restrictions (are non‐increasing) under the null of no p‐hacking. We find novel additional testable restrictions for p‐values based on t‐tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of p‐values. These testable restrictions result in more powerful tests for the null hypothesis of no p‐hacking. When there is also publication bias, our tests are joint tests for p‐hacking and publication bias. A reanalysis of two prominent data sets shows the usefulness of our new tests.

Keywords: p-values; p-curve; complete monotonicity; publication bias; Economic Theory; Applied Economics; Econometrics (search for similar items in EconPapers)
Date: 2022-01-01
New Economics Papers: this item is included in nep-sog
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.escholarship.org/uc/item/2p04s3dr.pdf;origin=repeccitec (application/pdf)

Related works:
Journal Article: Detecting p‐Hacking (2022)
Working Paper: Detecting p-hacking (2021)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cdl:ucsdec:qt2p04s3dr

Access Statistics for this paper

More papers in University of California at San Diego, Economics Working Paper Series from Department of Economics, UC San Diego Contact information at EDIRC.
Bibliographic data for series maintained by Lisa Schiff ().