Measuring probabilistic coherence to identify superior forecasters

Emily H. Ho, David V. Budescu and Mark Himmelstein

International Journal of Forecasting, 2025, vol. 41, issue 2, 596-612

Abstract: Forecasts, or subjective probability assessments of uncertain events, are characterized by two qualities: coherence, the degree to which the judgments are internally consistent, and correspondence, the extent to which judgments are accurate. Recent evidence suggests that more coherent forecasts tend to be more accurate. However, currently, there is no good stand-alone measure of probabilistic coherence. We developed and validated the Coherence Forecasting Scale (CFS). This questionnaire assesses how well people understand and apply probabilistic reasoning rules such as relations between joint and disjoint probabilities, probability complementarity, stochastic dominance, and monotonicity. In three incentivized forecasting tournaments, including one from an online public forecasting platform, judges who scored higher on the CFS were also more accurate. Notably, across all tournaments, the CFS dominates all administered individual difference and demographic measures in explanatory power predicting judgment accuracy, providing empirical evidence that coherence and accuracy are strongly linked.

Keywords: Accuracy; Coherence; Correspondence; Judgmental forecasting; Subjective probability judgements (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0169207024000104
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:intfor:v:41:y:2025:i:2:p:596-612

DOI: 10.1016/j.ijforecast.2024.02.005

Access Statistics for this article

International Journal of Forecasting is currently edited by R. J. Hyndman

More articles in International Journal of Forecasting from Elsevier
Bibliographic data for series maintained by Catherine Liu ().