Data Snooping Bias in Tests of the Relative Performance of Multiple Forecasting Models

Dan Gabriel Anghel

Journal of Banking & Finance, 2021, vol. 126, issue C

Abstract: Tests of the relative performance of multiple forecasting models are sensitive to how the set of alternatives is defined. Evaluating one model against a particular set may show that it has superior predictive ability. However, changing the number or type of alternatives in the set may demonstrate otherwise. This paper focuses on forecasting models based on technical analysis and analyzes how much data snooping bias can occur in tests from restricting the size of forecasting model “universes” or ignoring alternatives used by practitioners and other researchers. A Monte Carlo simulation shows that false discoveries have an average increase of 0.72-2.5 percentage points each time one removes half of the prediction models from the set of relevant alternatives. A complementary empirical investigation suggests that at least 50% of positive findings reported in the literature concerned with trading rule overperformance may be false. Our results motivate several recommendations for applied researchers that would alleviate data snooping bias in some of the more popular statistical tests used in the literature.

Keywords: Superior Predictive Ability; Data Snooping Bias; False Discoveries; Reality Check; Technical Analysis; Trading Rules; Efficient Market Hypothesis (search for similar items in EconPapers)
JEL-codes: C12 C18 G11 G14 G17 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378426621000716
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:jbfina:v:126:y:2021:i:c:s0378426621000716

DOI: 10.1016/j.jbankfin.2021.106113

Access Statistics for this article

Journal of Banking & Finance is currently edited by Ike Mathur

More articles in Journal of Banking & Finance from Elsevier
Bibliographic data for series maintained by Catherine Liu ().