In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments

Guillaume Hollard, Sébastien Massoni and Jean-Christophe Vergnaud

Post-Print from HAL

Abstract: In this paper, we use an experimental design to compare the performance of elicitation rules for subjective beliefs. Contrary to previous works in which elicited beliefs are compared to an objective benchmark, we consider a purely subjective belief framework (confidence in one's own performance in a cognitive task and a perceptual task). The performance of different elicitation rules is assessed according to the accuracy of stated beliefs in predicting success. We measure this accuracy using two main factors: calibration and discrimination. For each of them, we propose two statistical indexes and we compare the rules' performances for each measurement. The matching probability method provides more accurate beliefs in terms of discrimination, while the quadratic scoring rule reduces overconfidence and the free rule, a simple rule with no incentives, which succeeds in eliciting accurate beliefs. Nevertheless, the matching probability appears to be the best mechanism for eliciting beliefs due to its performances in terms of calibration and discrimination, but also its ability to elicit consistent beliefs across measures and across tasks, as well as its empirical and theoretical properties.

Keywords: Belief elicitation; Scoring rules; Confidence; Calibration; Discrimination; Incentives (search for similar items in EconPapers)
Date: 2016-03
References: Add references at CitEc
Citations: View citations in EconPapers (28)

Published in Theory and Decision, 2016, 80 (3), pp.363-387. ⟨10.1007/s11238-015-9509-9⟩

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
Working Paper: In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments (2016)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-01306258

DOI: 10.1007/s11238-015-9509-9

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().