Star Wars: The Empirics Strike Back

Abel Brodeur, Mathias Lé, Marc Sangnier and Yanos Zylberberg

No 1505E, Working Papers from University of Ottawa, Department of Economics

Abstract: Journals favor rejection of the null hypothesis. This selection upon tests may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a two humped camel shape with abundant p-values above 0.25, a valley between 0.25 and 0.10, and a bump slightly below 0.05. The missing tests (with p-values between 0.25 and 0.10) can be retrieved just after the 0.05 threshold and represent 10% to 20% of marginally rejected tests. Our interpretation is that researchers might be tempted to inflate the value of those just-rejected tests by choosing a significant specification. We propose a method to measure this residual and describe how it varies by article and author characteristics.

Keywords: Hypothesis testing; distorting incentives; selection bias; research in economics (search for similar items in EconPapers)
JEL-codes: A11 B41 C13 C44 (search for similar items in EconPapers)
Pages: 70 pages
Date: 2015
New Economics Papers: this item is included in nep-hme and nep-sog
References: Add references at CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://sciencessociales.uottawa.ca/economics/sites ... mics/files/1505e.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 502 Bad Gateway (http://sciencessociales.uottawa.ca/economics/sites/socialsciences.uottawa.ca.economics/files/1505e.pdf [301 Moved Permanently]--> https://sciencessociales.uottawa.ca/economics/sites/socialsciences.uottawa.ca.economics/files/1505e.pdf)

Related works:
Journal Article: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2015)
Working Paper: Star Wars: The Empirics Strike Back (2013)
Working Paper: Star wars: The empirics strike back (2012)
Working Paper: Star wars: The empirics strike back (2012)
Working Paper: Star wars: The empirics strike back (2012)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ott:wpaper:1505e

Access Statistics for this paper

More papers in Working Papers from University of Ottawa, Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Aggey Semenov (econ@uottawa.ca).