Aggregating Probabilistic Forecasts from Incoherent and Abstaining Experts

Joel B. Predd (), Daniel N. Osherson (), Sanjeev R. Kulkarni () and H. Vincent Poor ()
Additional contact information
Joel B. Predd: RAND Corporation, Pittsburgh, Pennsylvania 15213
Daniel N. Osherson: Department of Psychology, Princeton University, Princeton, New Jersey 08544
Sanjeev R. Kulkarni: Department of Electrical Engineering, Princeton University, Princeton, New Jersey 08544
H. Vincent Poor: Department of Electrical Engineering, Princeton University, Princeton, New Jersey 08544

Decision Analysis, 2008, vol. 5, issue 4, 177-189

Abstract: Decision makers often rely on expert opinion when making forecasts under uncertainty. In doing so, they confront two methodological challenges: the elicitation problem, which requires them to extract meaningful information from experts; and the aggregation problem, which requires them to combine expert opinion by resolving disagreements. Linear averaging is a justifiably popular method for addressing aggregation, but its robust simplicity makes two requirements on elicitation. First, each expert must offer probabilistically coherent forecasts; second, each expert must respond to all our queries. In practice, human judges (even experts) may be incoherent, and may prefer to assess only the subset of events about which they are comfortable offering an opinion. In this paper, a new methodology is developed for combining expert assessment of chance. The method retains the conceptual and computational simplicity of linear averaging, but generalizes the standard approach by relaxing the requirements on expert elicitation. The method also enjoys provable performance guarantees, and in experiments with real-world forecasting data is shown to offer both computational efficiency and competitive forecasting gains as compared to rival aggregation methods. This paper is relevant to the practice of decision analysis, for it enables an elicitation methodology in which judges have freedom to choose the events they assess.

Keywords: combining forecasts; probability forecasting; probability; incoherence; adjusting forecasts (search for similar items in EconPapers)
Date: 2008
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (17)

Downloads: (external link)
http://dx.doi.org/10.1287/deca.1080.0119 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:5:y:2008:i:4:p:177-189

Access Statistics for this article

More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().