Detecting p-hacking

Graham Elliott (), Nikolay Kudrin and Kaspar Wüthrich

Papers from arXiv.org

Abstract: We theoretically analyze the problem of testing for $p$-hacking based on distributions of $p$-values across multiple studies. We provide general results for when such distributions have testable restrictions (are non-increasing) under the null of no $p$-hacking. We find novel additional testable restrictions for $p$-values based on $t$-tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of $p$-values. These testable restrictions result in more powerful tests for the null hypothesis of no $p$-hacking. When there is also publication bias, our tests are joint tests for $p$-hacking and publication bias. A reanalysis of two prominent datasets shows the usefulness of our new tests.

Date: 2019-06, Revised 2021-05
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Published in Econometrica, Volume 90, Issue 2 (March 2022)

Downloads: (external link)
http://arxiv.org/pdf/1906.06711 Latest version (application/pdf)

Related works:
Journal Article: Detecting p‐Hacking (2022)
Working Paper: Detecting p‐Hacking (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:1906.06711

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().