Improved Tests for Forecast Comparisons in the Presence of Instabilities

Luis Martins and Pierre Perron

No wp2015-014, Boston University - Department of Economics - Working Papers Series from Boston University - Department of Economics

Abstract: Of interest is comparing the out-of-sample forecasting performance of two competing models in the presence of possible instabilities. To that effect, we suggest using simple structural change tests, sup-Wald and UDmax as proposed by Andrews (1993) and Bai and Perron (1998), for changes in the mean of the loss-differences. Giacomini and Rossi (2010) proposed a áuctuations test and a one-time reversal test also applied to the loss-differences. When properly constructed to account for potential serial correlation under the null hypothesis to have a pivotal limit distribution, it is shown that their tests have undesirable power properties, power that can be low and non-increasing as the alternative gets further from the null hypothesis. The good power properties they reported is simply an artifact of imposing a priori that the loss di§erentials are serially uncorrelated and using the simple sample variance to scale the tests. On the contrary, our statistics are shown to have higher monotonic power, especially the UDmax version. We use their empirical examples to show the practical relevance of the issues raised.

Keywords: non-monotonic power; structural change; forecasts; long-run variance (search for similar items in EconPapers)
JEL-codes: C22 C53 (search for similar items in EconPapers)
Date: 2015-10-06
New Economics Papers: this item is included in nep-ecm, nep-ets, nep-for and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://people.bu.edu/perron/papers/Forecast-note.pdf
Our link check indicates that this URL is bad, the error code is: 404 Not Found (http://people.bu.edu/perron/papers/Forecast-note.pdf [301 Moved Permanently]--> https://people.bu.edu/perron/papers/Forecast-note.pdf)

Related works:
Journal Article: Improved Tests for Forecast Comparisons in the Presence of Instabilities (2016)
Working Paper: Improved Tests for Forecast Comparisons in the Presence of Instabilities (2014)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bos:wpaper:wp2015-014

Access Statistics for this paper

More papers in Boston University - Department of Economics - Working Papers Series from Boston University - Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Program Coordinator ().