A Bayesian Approach to Backtest Overfitting

Jiří Witzany

No 2017/18, Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies

Abstract: Quantitative investment strategies are often selected from a broad class of candidate models estimated and tested on historical data. Standard statistical technique to prevent model overfitting such as out-sample back-testing turns out to be unreliable in the situation when selection is based on results of too many models tested on the holdout sample. There is an ongoing discussion how to estimate the probability of back-test overfitting and adjust the expected performance indicators like Sharpe ratio in order to reflect properly the effect of multiple testing. We propose a consistent Bayesian approach that consistently yields the desired robust estimates based on an MCMC simulation. The approach is tested on a class of technical trading strategies where a seemingly profitable strategy can be selected in the naive approach.

Keywords: Backtest; multiple testing; bootstrapping; cross-validation; probability of backtest overfitting; investment strategy; optimization; Sharpe ratio; Bayesian probability; MCMC (search for similar items in EconPapers)
JEL-codes: C11 C12 C5 C52 G1 G2 G24 (search for similar items in EconPapers)
Pages: 29 pages
Date: 2017-09, Revised 2017-09
New Economics Papers: this item is included in nep-ecm
References: Add references at CitEc
Citations:

Downloads: (external link)
http://ies.fsv.cuni.cz/sci/publication/show/id/5691/lang/cs (application/pdf)
Our link check indicates that this URL is bad, the error code is: 404 Not Found (http://ies.fsv.cuni.cz/sci/publication/show/id/5691/lang/cs [301 Moved Permanently]--> https://ies.fsv.cuni.cz/sci/publication/show/id/5691/lang/cs)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fau:wpaper:wp2017_18

Access Statistics for this paper

More papers in Working Papers IES from Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies Contact information at EDIRC.
Bibliographic data for series maintained by Natalie Svarcova ().