Regularized Aggregation of One-Off Probability Predictions

Ville A. Satopää ()
Additional contact information
Ville A. Satopää: Department of Technology and Operations Management, INSEAD, Boulevard de Constance, 77305 Fontainebleau CEDEX, France

Operations Research, 2022, vol. 70, issue 6, 3558-3580

Abstract: Forecasters predicting the chances of a future event may disagree because of differing evidence or noise. To harness the collective evidence of the crowd, we propose a Bayesian aggregator that is regularized by analyzing the forecasters’ disagreement and ascribing overdispersion to noise. Our aggregator requires no user intervention and can be computed efficiently even for a large number of predictions. To illustrate, we evaluate our aggregator on subjective probability predictions collected during a four-year forecasting tournament sponsored by the U.S. intelligence community. Our aggregator improves the squared error (a.k.a., the Brier score) of simple averaging by around 20% and other commonly used aggregators by 10%–25%. This advantage stems almost exclusively from improved calibration. An R package called braggR implements our method and is available on CRAN.

Keywords: Decision Analysis; judgmental forecasting; information aggregation; objective Bayes; overdispersion; wisdom of the crowds (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2021.2224 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:70:y:2022:i:6:p:3558-3580

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().