Evaluating Quantile Assessments

Victor Richmond R. Jose () and Robert L. Winkler ()
Additional contact information
Victor Richmond R. Jose: McDonough School of Business, Georgetown University, Washington, DC 20057
Robert L. Winkler: Fuqua School of Business, Duke University, Durham, North Carolina 27708

Operations Research, 2009, vol. 57, issue 5, 1287-1297

Abstract: Quantile assessments are commonly encountered in the elicitation of probability distributions in decision analysis, forecasting, and risk analysis. Scoring rules have been developed to provide ex ante incentives for careful and truthful assessments and ex post evaluation measures in the context of probability assessment. We show that these scoring rules designed for probability assessment provide inappropriate incentives if used for quantile assessment. We investigate the properties of a linear family of scoring rules that are intended specifically for quantile assessment (including the assessment of multiple quantiles) and can be related to a realistic decision-making problem. These rules provide proper incentives for quantile assessment and yield higher expected scores for distributions that are more informative in the sense of having less dispersion. We discuss the special case of interval forecasts and a generalization involving transformations, and we briefly mention other possible extensions.

Keywords: probability; assessment; evaluation; decision analysis; expert information; forecasting; probability forecasts (search for similar items in EconPapers)
Date: 2009
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (36)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.1080.0665 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:57:y:2009:i:5:p:1287-1297

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().