Two Reasons to Make Aggregated Probability Forecasts More Extreme

Jonathan Baron (), Barbara A. Mellers (), Philip E. Tetlock (), Eric Stone () and Lyle H. Ungar ()
Additional contact information
Jonathan Baron: Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
Barbara A. Mellers: Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
Philip E. Tetlock: Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
Eric Stone: Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
Lyle H. Ungar: Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania 19104

Decision Analysis, 2014, vol. 11, issue 2, 133-145

Abstract: When aggregating the probability estimates of many individuals to form a consensus probability estimate of an uncertain future event, it is common to combine them using a simple weighted average. Such aggregated probabilities correspond more closely to the real world if they are transformed by pushing them closer to 0 or 1. We explain the need for such transformations in terms of two distorting factors: The first factor is the compression of the probability scale at the two ends, so that random error tends to push the average probability toward 0.5. This effect does not occur for the median forecast, or, arguably, for the mean of the log odds of individual forecasts. The second factor---which affects mean, median, and mean of log odds---is the result of forecasters taking into account their individual ignorance of the total body of information available. Individual confidence in the direction of a probability judgment (high/low) thus fails to take into account the wisdom of crowds that results from combining different evidence available to different judges. We show that the same transformation function can approximately eliminate both distorting effects with different parameters for the mean and the median. And we show how, in principle, use of the median can help distinguish the two effects.

Keywords: forecasting; aggregation; wisdom of crowds; confidence (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (23)

Downloads: (external link)
http://dx.doi.org/10.1287/deca.2014.0293 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:11:y:2014:i:2:p:133-145

Access Statistics for this article

More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().