Detecting p‐Hacking

Graham Elliott (), Nikolay Kudrin and Kaspar Wüthrich

Econometrica, 2022, vol. 90, issue 2, 887-906

Abstract: We theoretically analyze the problem of testing for p‐hacking based on distributions of p‐values across multiple studies. We provide general results for when such distributions have testable restrictions (are non‐increasing) under the null of no p‐hacking. We find novel additional testable restrictions for p‐values based on t‐tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of p‐values. These testable restrictions result in more powerful tests for the null hypothesis of no p‐hacking. When there is also publication bias, our tests are joint tests for p‐hacking and publication bias. A reanalysis of two prominent data sets shows the usefulness of our new tests.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.3982/ECTA18583

Related works:
Working Paper: Detecting p‐Hacking (2022)
Working Paper: Detecting p-hacking (2021)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:emetrp:v:90:y:2022:i:2:p:887-906

Ordering information: This journal article can be ordered from
https://www.economet ... ordering-back-issues

Access Statistics for this article

Econometrica is currently edited by Guido W. Imbens

More articles in Econometrica from Econometric Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().