Choice of Sample Split in Out-of-Sample Forecast Evaluation

Peter Hansen and Allan Timmermann ()
Additional contact information
Allan Timmermann: UCSD and CREATES, Postal: Rady School of Management, 9500 Gilman Drive #0553, La Jolla, CA 92093-0553, USA

CREATES Research Papers from Department of Economics and Business Economics, Aarhus University

Abstract: Out-of-sample tests of forecast performance depend on how a given data set is split into estimation and evaluation periods, yet no guidance exists on how to choose the split point. Empirical forecast evaluation results can therefore be difficult to interpret, particularly when several values of the split point might have been considered. When the sample split is viewed as a choice variable, rather than being ?xed ex ante, we show that very large size distortions can occur for conventional tests of predictive accu- racy. Spurious rejections are most likely to occur with a short evaluation sample, while conversely the power of forecast evaluation tests is strongest with long out-of-sample periods. To deal with size distortions, we propose a test statistic that is robust to the effect of considering multiple sample split points. Empirical applications to predictabil- ity of stock returns and in?ation demonstrate that out-of-sample forecast evaluation results can critically depend on how the sample split is determined.

Keywords: Out-of-sample forecast evaluation; data mining; recursive estimation; predictability of stock returns; in?ation forecasting. (search for similar items in EconPapers)
JEL-codes: C12 C53 G17 (search for similar items in EconPapers)
Pages: 42
Date: 2012-02-07
New Economics Papers: this item is included in nep-ets and nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (82)

Downloads: (external link)
https://repec.econ.au.dk/repec/creates/rp/12/rp12_43.pdf (application/pdf)

Related works:
Working Paper: Choice of Sample Split in Out-of-Sample Forecast Evaluation (2012)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aah:create:2012-43

Access Statistics for this paper

More papers in CREATES Research Papers from Department of Economics and Business Economics, Aarhus University
Bibliographic data for series maintained by ().