EconPapers    
Economics at your fingertips  
 

Star Wars: The Empirics Strike Back

Abel Brodeur, Mathias Lé (), Marc Sangnier () and Yanos Zylberberg ()

PSE Working Papers from HAL

Abstract: Journals favor rejection of the null hypothesis. This selection upon tests may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a two humped camel shape with abundant p-values above 0.25, a valley between 0.25 and 0.10, and a bump slightly below 0.05. The missing tests (with p-values between 0.25 and 0.10) can be retrieved just after the 0.05 threshold and represent 10% to 20% of marginally rejected tests. Our interpretation is that researchers might be tempted to inflate the value of those just-rejected tests by choosing a "significant" specification. We propose a method to measure this residual and describe how it varies by article and author characteristics.

Keywords: selection bias; research in economics; distorting incentives; hypothesis testing (search for similar items in EconPapers)
Date: 2015-05
New Economics Papers: this item is included in nep-hpe
Note: View the original document on HAL open archive server: https://halshs.archives-ouvertes.fr/halshs-01158500
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2) Track citations by RSS feed

Downloads: (external link)
https://halshs.archives-ouvertes.fr/halshs-01158500/document (application/pdf)

Related works:
Journal Article: Star Wars: The Empirics Strike Back (2016) Downloads
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2016)
Working Paper: Star Wars: The Empirics Strike Back (2015) Downloads
Working Paper: Star Wars: The Empirics Strike Back (2015) Downloads
Working Paper: Star Wars: The Empirics Strike Back (2015) Downloads
Working Paper: Star Wars: The Empirics Strike Back (2013) Downloads
Working Paper: Star wars: The empirics strike back (2012) Downloads
Working Paper: Star wars: The empirics strike back (2012) Downloads
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:psewpa:halshs-01158500

Access Statistics for this paper

More papers in PSE Working Papers from HAL
Bibliographic data for series maintained by CCSD ().

 
Page updated 2021-01-26
Handle: RePEc:hal:psewpa:halshs-01158500