Star wars: The empirics strike back
Abel Brodeur,
Mathias Lé,
Marc Sangnier and
Yanos Zylberberg
Working Papers from HAL
Abstract:
Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above :25, a valley between :25 and :10 and a bump slightly under :05. Missing tests are those which would have been accepted but close to being rejected (p-values between :25 and :10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10% and 20% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models.
Keywords: Hypothesis testing; Distorting incentives; Selection bias; Research in economics (search for similar items in EconPapers)
Date: 2012-06
Note: View the original document on HAL open archive server: https://shs.hal.science/halshs-00710122v1
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)
Downloads: (external link)
https://shs.hal.science/halshs-00710122v1/document (application/pdf)
Related works:
Journal Article: Star Wars: The Empirics Strike Back (2016) 
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2015) 
Working Paper: Star Wars: The Empirics Strike Back (2015) 
Working Paper: Star Wars: The Empirics Strike Back (2015) 
Working Paper: Star Wars: The Empirics Strike Back (2015) 
Working Paper: Star Wars: The Empirics Strike Back (2013) 
Working Paper: Star wars: The empirics strike back (2012) 
Working Paper: Star wars: The empirics strike back (2012) 
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:hal:wpaper:halshs-00710122
Access Statistics for this paper
More papers in Working Papers from HAL
Bibliographic data for series maintained by CCSD ().