Scoring Probability Forecasts by a User’s Bets Against a Market Consensus

David Johnstone (), Stewart Jones (), Oliver Jones and Steve Tulig
Additional contact information
David Johnstone: Faculty of Business, University of Wollongong, Wollongong, New South Wales 2500, Australia
Stewart Jones: University of Sydney Business School, University of Sydney, Sydney 2006, Australia
Oliver Jones: Faculty of Mathematics, East Los Angeles College, Monterey Park, California 91754
Steve Tulig: Faculty of Business, University of Wollongong, Wollongong, New South Wales 2500, Australia

Decision Analysis, 2021, vol. 18, issue 3, 169-184

Abstract: The purpose of our paper is to describe a probability scoring rule that reflects the economic performance of a hypothetical investor who acts upon the probability forecasts emanating from a given model or human expert by trading against a market-clearing consensus of competing models and forecasts. The probability forecasts being compared are aggregated by an equilibrium condition into a market consensus reflecting the wisdom of the crowd. A good forecaster (model or human expert) is measured as one who allows the user to bet profitably against the market consensus. By asking forecasts to beat the market, forecasters are discouraged from herding and motivated to obtain better information than rival forecasters. We illustrate and prove that each trader’s personal incentive to hedge or fudge disappears when the number of forecasts in the market is sufficiently large. Our score exhibits the forecaster’s ability to assist economically profitable action and reveals how the user’s profits depend strongly on the accuracy of the forecasts and the decision rule (boldness) with which they are acted upon.

Keywords: probability forecast; probability of bankruptcy; prediction markets; forecast evaluation; economic scoring rules (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/deca.2020.0424 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:18:y:2021:i:3:p:169-184

Access Statistics for this article

More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().