A study of expert overconfidence

Shi-Woei Lin and Vicki M. Bier

Reliability Engineering and System Safety, 2008, vol. 93, issue 5, 711-721

Abstract: Overconfidence is one of the most common (and potentially severe) problems in expert judgment. To assess the extent of expert overconfidence, we analyzed a large data set on expert opinion compiled by Cooke and colleagues at the Technical University of Delft and elsewhere. This data set contains roughly five thousand 90% confidence intervals of uncertain quantities for which the true values are now known. Our analysis assesses the overall extent of overconfidence in the data set. Significant differences in the extent of overconfidence were found among studies, among experts, and among questions within a study. Moreover, replications (multiple realizations for the same question) allowed a preliminary assessment of whether the question effect is due largely to question difficulty, or merely to random noise in the realizations of the uncertain quantities. The results of this analysis suggest that much of the apparent question effect may be due to noise rather than systematic differences in the difficulty of achieving good calibration for different questions. The results support the differential weighting of experts, since there are significant differences in expert calibration within studies.

Keywords: Expert opinion; Calibration; Overconfidence; Expert aggregation (search for similar items in EconPapers)
Date: 2008
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (26)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832007000981
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:93:y:2008:i:5:p:711-721

DOI: 10.1016/j.ress.2007.03.014

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().