Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs

Yael Grushka-Cockayne (), Kenneth C. Lichtendahl (), Victor Richmond R. Jose () and Robert L. Winkler ()
Additional contact information
Yael Grushka-Cockayne: Darden School of Business, University of Virginia, Charlottesville, Virginia 22903
Kenneth C. Lichtendahl: Darden School of Business, University of Virginia, Charlottesville, Virginia 22903
Victor Richmond R. Jose: McDonough School of Business, Georgetown University, Washington, DC 20057
Robert L. Winkler: The Fuqua School of Business, Duke University, Durham, North Carolina 27708

Operations Research, 2017, vol. 65, issue 3, 712-728

Abstract: From forecasting competitions to conditional value-at-risk requirements, the use of multiple quantile assessments is growing in practice. To evaluate them, we use a rule from the general class of proper scoring rules for a forecaster’s multiple quantiles of a single uncertain quantity of interest. The general rule is additive in the component scores. Each component contains a function that measures its quantile’s distance from the realization and weights its contribution to the overall score. To determine this function, we propose that the score of a group’s combined quantile should be better than that of a randomly selected forecaster’s quantile only when the forecasters bracket the realization (i.e., their quantiles do not fall on the same side of the realization). If a score satisfies this property, we say it is sensitive to bracketing. We characterize the class of proper scoring rules that is sensitive to bracketing when the decision maker uses a generalized average to combine forecasters’ quantiles. Finally, we show how weights can be set to match the payoffs in many important business contexts.

Keywords: quantile forecasts; forecast evaluation; probability elicitation; proper scoring rules; expert aggregation; expert combination (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
https://doi.org/10.1287/opre.2017.1588 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:65:y:2017:i:3:p:712-728

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().