Scores for Multivariate Distributions and Level Sets

Xiaochun Meng (), James W. Taylor (), Souhaib Ben Taieb () and Siran Li ()
Additional contact information
Xiaochun Meng: University of Sussex Business School, University of Sussex, Brighton BN1 9SN, United Kingdom
James W. Taylor: Saïd Business School, University of Oxford, Oxford OX1 1HP, United Kingdom
Souhaib Ben Taieb: Department of Computer Science, University of Mons, 7000 Mons, Belgium
Siran Li: School of Mathematical Sciences and IMA-Shanghai, Shanghai Jiao Tong University, Shanghai 200240, China

Operations Research, 2025, vol. 73, issue 1, 344-362

Abstract: Forecasts of multivariate probability distributions are required for a variety of applications. Scoring rules enable the evaluation of forecast accuracy and comparison between forecasting methods. We propose a theoretical framework for scoring rules for multivariate distributions that encompasses the existing quadratic score and multivariate continuous ranked probability score. We demonstrate how this framework can be used to generate new scoring rules. In some multivariate contexts, it is a forecast of a level set that is needed, such as a density level set for anomaly detection or the level set of the cumulative distribution as a measure of risk. This motivates consideration of scoring functions for such level sets. For univariate distributions, it is well established that the continuous ranked probability score can be expressed as the integral over a quantile score. We show that, in a similar way, scoring rules for multivariate distributions can be decomposed to obtain scoring functions for level sets. Using this, we present scoring functions for different types of level sets, including density level sets and level sets for cumulative distributions. To compute the scores, we propose a simple numerical algorithm. We perform a simulation study to support our proposals, and we use real data to illustrate usefulness for forecast combining and conditional value at risk estimation.

Keywords: Decision Analysis; decision analysis; probabilistic forecasts; scoring functions; multivariate probability distributions; level sets; quantiles (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2020.0365 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:73:y:2025:i:1:p:344-362

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().