Incentive-Compatible Forecasting Competitions

Jens Witkowski (), Rupert Freeman (), Jennifer Wortman Vaughan (), David M. Pennock () and Andreas Krause ()
Additional contact information
Jens Witkowski: Frankfurt School of Finance & Management, Frankfurt 60322, Germany
Rupert Freeman: Darden School of Business, University of Virginia, Charlottesville, Virginia 22903
Jennifer Wortman Vaughan: Microsoft Research, New York, New York 10012
David M. Pennock: Department of Computer Science, Rutgers University, New Brunswick, New Jersey 08854
Andreas Krause: Department of Computer Science, ETH Zurich, Zurich 8092, Switzerland

Management Science, 2023, vol. 69, issue 3, 1354-1374

Abstract: We initiate the study of incentive-compatible forecasting competitions in which multiple forecasters make predictions about one or more events and compete for a single prize. We have two objectives: (1) to incentivize forecasters to report truthfully and (2) to award the prize to the most accurate forecaster. Proper scoring rules incentivize truthful reporting if all forecasters are paid according to their scores. However, incentives become distorted if only the best-scoring forecaster wins a prize, since forecasters can often increase their probability of having the highest score by reporting more extreme beliefs. In this paper, we introduce two novel forecasting competition mechanisms. Our first mechanism is incentive compatible and guaranteed to select the most accurate forecaster with probability higher than any other forecaster. Moreover, we show that in the standard single-event, two-forecaster setting and under mild technical conditions, no other incentive-compatible mechanism selects the most accurate forecaster with higher probability. Our second mechanism is incentive compatible when forecasters’ beliefs are such that information about one event does not lead to belief updates on other events, and it selects the best forecaster with probability approaching one as the number of events grows. Our notion of incentive compatibility is more general than previous definitions of dominant strategy incentive compatibility in that it allows for reports to be correlated with the event outcomes. Moreover, our mechanisms are easy to implement and can be generalized to the related problems of outputting a ranking over forecasters and hiring a forecaster with high accuracy on future events.

Keywords: artificial intelligence; forecasting; economics: game theory and bargaining (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2022.4410 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:69:y:2023:i:3:p:1354-1374

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().