On Forecast Evaluation

Wilmer Martinez-Rivera (), Manuel Hernandez-Bejarano and Juan Julio

No 11604, Borradores de Economia from Banco de la Republica

Abstract: We propose to assess the performance of k forecast procedures by exploring the distributions of forecast errors and error losses. We argue that non systematic forecast errors minimize when their distributions are symmetric and unimodal, and that forecast accuracy should be assessed through stochastic loss order rather than expected loss order, which is the way it is customarily performed in previous work. Moreover, since forecast performance evaluation can be understood as a one way analysis of variance, we propose to explore loss distributions under two circumstances; when a strict (but unknown) joint stochastic order exists among the losses of all forecast alternatives, and when such order happens among subsets of alternative procedures. In spite of the fact that loss stochastic order is stronger than loss moment order, our proposals are at least as powerful as competing tests, and are robust to the correlation, autocorrelation and heteroskedasticity settings they consider. In addition, since our proposals do not require samples of the same size, their scope is also wider, and provided that they test the whole loss distribution instead of just loss moments, they can also be used to study forecast distributions as well. We illustrate the usefulness of our proposals by evaluating a set of real world forecasts.

Keywords: Forecast evaluation; Stochastic order; Multiple comparison. (search for similar items in EconPapers)
JEL-codes: C12 C14 C53 (search for similar items in EconPapers)
Pages: 27
Date: 2014-06-06
New Economics Papers: this item is included in nep-ecm and nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.banrep.gov.co/sites/default/files/publicaciones/archivos/be_825.pdf

Related works:
Working Paper: On Forecast Evaluation (2014)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:col:000094:011604

Access Statistics for this paper

More papers in Borradores de Economia from Banco de la Republica
Bibliographic data for series maintained by Clorith Angelica Bahos Olivera ().