Comparing input interfaces to elicit belief distributions

Paolo Crosetto and Thomas De Haan

Working Papers from Grenoble Applied Economics Laboratory (GAEL)

Abstract: We develop an intuitive, Click-and-Drag interface to elicit continuous belief distributions of any shape. We test this interface against the state of the art in the experimental literature – a text-based interface and multiple sliders – and in the online forecasting industry – a distribution-manipulation interface similar to the one used at Metaculus, a crowd-forecasting website. By means of a pre-registered experiment on Amazon Mechanical Turk we collect quantitative data on the convergence speed and accuracy of reported beliefs in a series of induced-value scenarios varying by granularity, shape, and time constraints. We also collect subjective data on ease of use, frustration and understanding. Results show that the click-and-drag interface outperforms all others by accuracy and speed, and is self-reported as being more intuitive and less frustrating than other interfaces, confirming our pre-registered hypothesis. Besides pre-registration, we report that the click-and-drag interface generates the least drop-out rate from the task, and scores best in a sentiment analysis of an open-ended general question. Further, we use the interfaces to collect homegrown beliefs on temperature in New York City in 2022 and 2042. On average, all subjects overshoot the real temperature for 2022 by about 2°F, and all anticipate further global warming in the order of 2.3°F; these forecasts are by and large not impacted by the interface used to elicit them. We provide a free and open source, ready to use oTree and Qualtrics plugin of our click-and-drag and all other tested interfaces available at https://beliefelicitation.github.io/.

Keywords: Belief Elicitation; Forecasting; Scoring Rules; Interfaces (search for similar items in EconPapers)
JEL-codes: C88 C91 (search for similar items in EconPapers)
Pages: 26 pages
Date: 2022-11
New Economics Papers: this item is included in nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://gael.univ-grenoble-alpes.fr/sites/default/ ... 2022/gael2022-01.pdf (application/pdf)

Related works:
Working Paper: Comparing input interfaces to elicit belief distributions (2023)
Working Paper: Comparing input interfaces to elicit belief distributions (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gbl:wpaper:2022-01

Access Statistics for this paper

More papers in Working Papers from Grenoble Applied Economics Laboratory (GAEL) Contact information at EDIRC.
Bibliographic data for series maintained by Adrien Hervouet ().