Trimmed Opinion Pools and the Crowd's Calibration Problem

Victor Richmond R. Jose (), Yael Grushka-Cockayne () and Kenneth C. Lichtendahl ()
Additional contact information
Victor Richmond R. Jose: McDonough School of Business, Georgetown University, Washington, DC 20057
Yael Grushka-Cockayne: Darden School of Business, University of Virginia, Charlottesville, Virginia 22906
Kenneth C. Lichtendahl: Darden School of Business, University of Virginia, Charlottesville, Virginia 22906

Management Science, 2014, vol. 60, issue 2, 463-475

Abstract: We introduce an alternative to the popular linear opinion pool for combining individual probability forecasts. One of the well-known problems with the linear opinion pool is that it can be poorly calibrated. It tends toward underconfidence as the crowd's diversity increases, i.e., as the variance in the individuals' means increases. To address this calibration problem, we propose the exterior-trimmed opinion pool. To form this pool, forecasts with low and high means, or cumulative distribution function (cdf) values, are trimmed away from a linear opinion pool. Exterior trimming decreases the pool's variance and improves its calibration. A linear opinion pool, however, will remain overconfident when individuals are overconfident and not very diverse. For these situations, we suggest trimming away forecasts with moderate means or cdf values. This interior trimming increases variance and reduces overconfidence. Using probability forecast data from U.S. and European Surveys of Professional Forecasters, we present empirical evidence that trimmed opinion pools can outperform the linear opinion pool. This paper was accepted by Rakesh Sarin, decision analysis.

Keywords: trimming; probability forecasts; expert combination; linear opinion pool; underconfidence; overconfidence; scoring rules; wisdom of crowds (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2013.1781 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:60:y:2014:i:2:p:463-475

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().