Validating the Contribution-Weighted Model: Robustness and Cost-Benefit Analyses

Eva Chen (), David V. Budescu (), Shrinidhi K. Lakshmikanth (), Barbara A. Mellers () and Philip E. Tetlock ()
Additional contact information
Eva Chen: University of Pennsylvania, Philadelphia, Pennsylvania 19104
David V. Budescu: Department of Psychology, Fordham University, Bronx, New York 10458
Shrinidhi K. Lakshmikanth: University of Pennsylvania, Philadelphia, Pennsylvania 19104
Barbara A. Mellers: University of Pennsylvania, Philadelphia, Pennsylvania 19104
Philip E. Tetlock: University of Pennsylvania, Philadelphia, Pennsylvania 19104

Decision Analysis, 2016, vol. 13, issue 2, 128-152

Abstract: We use results from a multiyear, geopolitical forecasting tournament to highlight the ability of the contribution weighted model [Budescu DV, Chen E (2015) Identifying expertise to extract the wisdom of crowds. Management Sci. 61(2):267–280] to capture and exploit expertise. We show that the model performs better when judges gain expertise from manipulations such as training in probabilistic reasoning and collaborative interaction from serving on teams. We document the model’s robustness using probability judgments from early, middle, and late phases of the forecasting period and by showing its strong performance in the presence of hypothetical malevolent forecasters. The model is highly cost-effective: it operates well, even with random attrition, as the number of judges shrinks and information on their past performance is reduced.

Keywords: forecasts: combining; probability: weighting; probability: group; forecasts: aggregating; expertise; cost-benefit analysis (search for similar items in EconPapers)
Date: 2016
References: Add references at CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://dx.doi.org/10.1287/deca.2016.0329 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:13:y:2016:i:2:p:128-152

Access Statistics for this article

More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().