Forecast performance in the ECB SPF: ability or chance?

Aidan Meyler

No 2371, Working Paper Series from European Central Bank

Abstract: In this paper, we consider whether differences in the forecast performance of ECB SPF respondents reflect ability or chance. Although differences in performance metrics sometimes appear substantial, it is challenging to determine whether they reflect ex ante skill or other factors impacting ex post sampling variation such as the nature of economic shocks that materialised or simply which rounds participants responded in. We apply and adapt an approach developed by D’Agostino et al. (2012) who used US SPF data. They developed a test of a null hypothesis that all forecasters have equal ability. Their statistic reflects both the absolute and relative performance of each forecaster and they used bootstrap techniques to compare the empirical results with the equivalents obtained under the null hypothesis of equal forecaster ability. Our results, at a first pass, suggest that there would appear to be evidence of good/bad forecasters. However once we control for the autocorrelation that is caused by the overlapping rolling horizons, we find, like D’Agostino et al. (2012), that the best forecasters are not statistically significantly better than others. Unlike D’Agostino et al. (2012), however, we do not find evidence of forecasters that perform very significantly worse than others. Controlling for autocorrelation is a key feature of this paper relative to previous work. Our results hold considering the whole sample period of the ECB SPF (1999-2018) as well as the pre- and post-global financial crisis samples. We also find that when assessed across all variables and horizons, the aggregate (consensus) SPF forecast performs best. JEL Classification: C53, E27, E37

Keywords: bootstrap; forecasting; performance (search for similar items in EconPapers)
Date: 2020-02
New Economics Papers: this item is included in nep-eec and nep-for
Note: 496790
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp2371~4edce8ed72.en.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ecb:ecbwps:20202371

Access Statistics for this paper

More papers in Working Paper Series from European Central Bank 60640 Frankfurt am Main, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Official Publications ().