Comparing input interfaces to elicit belief distributions

Paolo Crosetto and Thomas de Haan
Additional contact information
Thomas de Haan: UiB - University of Bergen

Post-Print from HAL

Abstract: This paper introduces a new software interface to elicit belief distributions of any shape: Click-and-Drag . The interface was tested against the state of the art in the experimental literature—a text-based interface and multiple sliders—and in the online forecasting industry—a distribution-manipulation interface similar to the one used by the most popular crowd-forecasting website. By means of a pre-registered experiment on Amazon Mechanical Turk, quantitative data on the accuracy of reported beliefs in a series of induced-value scenarios varying by granularity, shape, and time constraints, as well as subjective data on user experience were collected. Click-and-Drag outperformed all other interfaces by accuracy and speed, and was self-reported as being more intuitive and less frustrating, confirming the pre-registered hypothesis. Aside of the pre-registered results, Click-and-Drag generated the least drop-out rate from the task, and scored best in a sentiment analysis of an open-ended general question. Further, the interface was used to collect homegrown predictions on temperature in New York City in 2022 and 2042. Click-and-Drag elicited distributions were smoother with less idiosyncratic spikes. Free and open source, ready to use oTree, Qualtrics and Limesurvey plugins for Click-and-Drag, and all other tested interfaces are available at https://beliefelicitation.github.io/ .

Keywords: Belief elicitation; Forecasting; Scoring rules; Interfaces (search for similar items in EconPapers)
Date: 2023
Note: View the original document on HAL open archive server: https://hal.science/hal-04263759v1
References: Add references at CitEc
Citations:

Published in Judgment and Decision Making, 2023, 18, pp.e27. ⟨10.1017/jdm.2023.21⟩

Downloads: (external link)
https://hal.science/hal-04263759v1/document (application/pdf)

Related works:
Working Paper: Comparing input interfaces to elicit belief distributions (2022)
Working Paper: Comparing input interfaces to elicit belief distributions (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-04263759

DOI: 10.1017/jdm.2023.21

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().