Improving Accuracy by Coherence Weighting of Direct and Ratio Probability Judgments

Yuyu Fan (), David V. Budescu (), David Mandel () and Mark Himmelstein ()
Additional contact information
Yuyu Fan: Data Science, AllianceBernstein LP, New York, New York 10105
David V. Budescu: Department of Psychology, Fordham University, Bronx, New York 10458
David Mandel: Intelligence, Influence and Collaboration Section, Toronto Research Centre, Defence Research and Development, Toronto, Ontario M3M 3B9, Canada
Mark Himmelstein: Department of Psychology, Fordham University, Bronx, New York 10458

Decision Analysis, 2019, vol. 16, issue 3, 197-217

Abstract: Human forecasts and other probabilistic judgments can be improved by elicitation and aggregation methods. Recent work on elicitation shows that deriving probability estimates from relative judgments (the ratio method) is advantageous, whereas other recent work on aggregation shows that it is beneficial to transform probabilities into coherent sets (coherentization) and to weight judges' assessments by their degree of coherence. We report an experiment that links these areas by examining the effect of coherentization and multiple forms of coherence weighting using direct and ratio elicitation methods on accuracy of probability judgments (both forecasts and events with known distributions). We found that coherentization invariably yields improvements to accuracy. Moreover, judges' levels of probabilistic coherence are related to their judgment accuracy. Therefore, coherence weighting can improve judgment accuracy, but the strength of the effect varies among elicitation and weighting methods. As well, the benefit of coherence weighting is stronger on “calibration” items that served as a basis for establishing the weights than for unrelated “test” items. Finally, echoing earlier research, we found overconfidence in judgment, and the degree of overconfidence was comparable between the two elicitation methods.

Keywords: subjective probability; coherentization; coherence weighting; probability elicitation; forecasting (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
https://doi.org/10.1287/deca.2018.0388 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:16:y:2019:i:3:p:197-217

Access Statistics for this article

More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().