 Should Scoring Rules be "Effective"?Robert F. Nau
Management Science, 1985, vol. 31, issue 5, 527-535 Abstract: A scoring rule is a reward function for eliciting or evaluating forecasts expressed as discrete or continuous probability distributions. A rule is strictly proper if it encourages the forecaster to state his true subjective probabilities, and effective if it is associated with a metric on the set of probability distributions. Recently, the property of effectiveness (which is stronger than strict properness) has been proposed as a desideratum for scoring rules for continuous forecasts, for reasons of "monotonicity" in keeping the forecaster close to his true probabilities, since in practice the forecast must be chosen from a low-dimensional set of "admissible" distributions. It is shown in this paper that what effectiveness implies, beyond strict properness, is not a monotonicity property but a transitivity property, which is difficult to justify behaviorally. The logarithmic scoring rule is shown to violate the transitivity property, and hence is not effective. The L 1 and L \infty metrics are shown to allow no effective scoring rules. Some potential difficulties in interpreting admissible forecasts are also discussed. Keywords: probability forecasting; evaluation of forecasts; proper scoring rules; effectiveness of scoring rules (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:31:y:1985:i:5:p:527-535 Access Statistics for this article More articles in Management Science from INFORMS Contact information at EDIRC. |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.