Star Wars: The Empirics Strike Back

Abel Brodeur, Mathias Lé, Marc Sangnier and Yanos Zylberberg

No 7268, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Journals favor rejection of the null hypothesis. This selection upon tests may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above 0.25, a valley between 0.25 and 0.10 and a bump slightly below 0.05. The missing tests (with p-values between 0.25 and 0.10) can be retrieved just after the 0.05 threshold and represent 10% to 20% of marginally rejected tests. Our interpretation is that researchers might be tempted to inflate the value of those almost-rejected tests by choosing a "significant" specification. We propose a method to measure inflation and decompose it along articles' and authors' characteristics.

Keywords: selection bias; distorting incentives; hypothesis testing; research in economics (search for similar items in EconPapers)
JEL-codes: A11 B41 C13 C44 (search for similar items in EconPapers)
Pages: 61 pages
Date: 2013-03
New Economics Papers: this item is included in nep-ecm and nep-sog
References: Add references at CitEc
Citations: View citations in EconPapers (46)

Published - published in: American Economic Journal: Applied Economics, 2016, 8 (1), 1-32

Downloads: (external link)
https://docs.iza.org/dp7268.pdf (application/pdf)

Related works:
Journal Article: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star wars: The empirics strike back (2012)
Working Paper: Star wars: The empirics strike back (2012)
Working Paper: Star wars: The empirics strike back (2012)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp7268

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().