p-Hacking: Evidence from Two Million Trading Strategies

Tarun Chordia, Amit Goyal and Alessio Saretto
Additional contact information
Tarun Chordia: Emory University - Department of Finance

No 17-37, Swiss Finance Institute Research Paper Series from Swiss Finance Institute

Abstract: We implement a data mining approach to generate about 2.1 million trading strategies. This large set of strategies serves as a laboratory to evaluate the seriousness of p-hacking and data snooping in finance. We apply multiple hypothesis testing techniques that account for cross-correlations in signals and returns to produce t-statistic thresholds that control the proportion of false discoveries. We find that the difference in rejections rates produced by single and multiple hypothesis testing is such that most rejections of the null of no outperformance under single hypothesis testing are likely false (i.e., we find a very high rate of type I errors). Combining statistical criteria with economic considerations, we find that a remarkably small number of strategies survive our thorough vetting procedure. Even these surviving strategies have no theoretical underpinnings. Overall, p-hacking is a serious problem and, correcting for it, outperforming trading strategies are rare.

Keywords: Hypothesis testing; False discoveries; Trading strategies (search for similar items in EconPapers)
JEL-codes: G10 G11 G12 (search for similar items in EconPapers)
Pages: 53 pages
Date: 2017-08, Revised 2018-04
New Economics Papers: this item is included in nep-ecm
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3017677 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:chf:rpseri:rp1737

Access Statistics for this paper

More papers in Swiss Finance Institute Research Paper Series from Swiss Finance Institute Contact information at EDIRC.
Bibliographic data for series maintained by Ridima Mittal ().