Score-based calibration testing for multivariate forecast distributions

Malte Knüppel, Fabian Krüger and Marc-Oliver Pohle

No 50/2022, Discussion Papers from Deutsche Bundesbank

Abstract: Multivariate distributional forecasts have become widespread in recent years. To assess the quality of such forecasts, suitable evaluation methods are needed. In the univariate case, calibration tests based on the probability integral transform (PIT) are routinely used. However, multivariate extensions of PIT-based calibration tests face various challenges. We therefore introduce a general framework for calibration testing in the multivariate case and propose two new tests that arise from it. Both approaches use proper scoring rules and are simple to implement even in large dimensions. The first employs the PIT of the score. The second is based on comparing the expected performance of the forecast distribution (i.e., the expected score) to its actual performance based on realized observations (i.e., the realized score). The tests have good size and power properties in simulations and solve various problems of existing tests. We apply the new tests to forecast distributions for macroeconomic and financial time series data.

Keywords: Forecast Evaluation; Density Forecasts; Ensemble Forecasts (search for similar items in EconPapers)
JEL-codes: C12 C52 C53 (search for similar items in EconPapers)
Date: 2022
New Economics Papers: this item is included in nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/268250/1/1830345494.pdf (application/pdf)

Related works:
Working Paper: Score-based calibration testing for multivariate forecast distributions (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:bubdps:502022

Access Statistics for this paper

More papers in Discussion Papers from Deutsche Bundesbank Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().