A Bunch of Models, a Bunch of Nulls and Inference About Predictive Ability

Pablo Pincheira

Working Papers Central Bank of Chile from Central Bank of Chile

Abstract: Inference about predictive ability is usually carried out in the form of pairwise comparisons between two competing forecasting methods. Nevertheless, some interesting questions are concerned with families of models and not just with a couple of forecasting strategies. An example of this would be the question about the predictive accuracy of pure time-series models versus models based on economic fundamentals. It is clear that an appropriate answer to this question requires comparing families of models, which may include a number of different forecasting strategies. Another usual approach in the literature consists of comparing the accuracy of a new forecasting method with a natural benchmark. Nevertheless, unless the econometrician is completely sure about the superiority of the benchmark over the rest of the methods available in the literature, he/she may want to compare the accuracy of his/her new forecasting model, and its extensions, against a broader set of methods. In this article we present a simple methodology to test the null hypothesis of equal predictive ability between two families of forecasting methods. Our approach corresponds to a natural extension of the White (2000) reality check in which we allow for the families being compared to be populated by a large number of forecasting methods. We illustrate our testing approach with an empirical application comparing the ability of two families of models to predict headline inflation in Chile, the US, Sweden and Mexico. With this illustration we show that comparing families of models using the usual approach based on pairwise comparisons of the best ex-post performing models in each family, may lead to conclusions that are at odds with those suggested by our approach.

Date: 2011-01
New Economics Papers: this item is included in nep-cba, nep-ecm and nep-for
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://www.bcentral.cl/documents/33528/133326/DTBC_607.pdf (application/pdf)

Related works:
Journal Article: A Bunch of Models, a Bunch of Nulls and Inference about Predictive Ability (2013)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:chb:bcchwp:607

Access Statistics for this paper

More papers in Working Papers Central Bank of Chile from Central Bank of Chile Contact information at EDIRC.
Bibliographic data for series maintained by Alvaro Castillo ().