A new approach for detecting shifts in forecast accuracy

Chiu, Ching-Wai (Jeremy), Simon Hayes, George Kapetanios and Konstantinos Theodoridis

International Journal of Forecasting, 2019, vol. 35, issue 4, 1596-1612

Abstract: Forecasts play a critical role at inflation-targeting central banks, such as the Bank of England. Breaks in the forecast performance of a model can potentially incur important policy costs. However, commonly-used statistical procedures implicitly place a lot of weight on type I errors (or false positives), which results in a relatively low power of the tests to identify forecast breakdowns in small samples. We develop a procedure which aims to capture the policy cost of missing a break. We use data-based rules to find the test size that optimally trades off the costs associated with false positives with those that can result from a break going undetected for too long. In so doing, we also explicitly study forecast errors as a multivariate system. The covariance between forecast errors for different series, although often overlooked in the forecasting literature, not only enables us to consider testing in a multivariate setting, but also increases the test power. As a result, we can tailor our choice of the critical values for each series not only to the in-sample properties of each series, but also to the way in which the series of forecast errors covary.

Keywords: Forecast breaks; Statistical decision making; Optimal test sizes; Hypothesis testing with small sample sizes; Central banking (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0169207019300706
Full text for ScienceDirect subscribers only

Related works:
Working Paper: A new approach for detecting shifts in forecast accuracy (2018)
Working Paper: A New Approach for Detecting Shifts in Forecast Accuracy (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:intfor:v:35:y:2019:i:4:p:1596-1612

DOI: 10.1016/j.ijforecast.2019.01.008

Access Statistics for this article

International Journal of Forecasting is currently edited by R. J. Hyndman

More articles in International Journal of Forecasting from Elsevier
Bibliographic data for series maintained by Catherine Liu ().