Evaluating probabilistic forecasts of football matches: the case against the ranked probability score

Edward Wheatcroft

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: A scoring rule is a function of a probabilistic forecast and a corresponding outcome used to evaluate forecast performance. There is some debate as to which scoring rules are most appropriate for evaluating forecasts of sporting events. This paper focuses on forecasts of the outcomes of football matches. The ranked probability score (RPS) is often recommended since it is 'sensitive to distance', that is it takes into account the ordering in the outcomes (a home win is 'closer' to a draw than it is to an away win). In this paper, this reasoning is disputed on the basis that it adds nothing in terms of the usual aims of using scoring rules. A local scoring rule is one that only takes the probability placed on the outcome into consideration. Two simulation experiments are carried out to compare the performance of the RPS, which is non-local and sensitive to distance, the Brier score, which is non-local and insensitive to distance, and the Ignorance score, which is local and insensitive to distance. The Ignorance score outperforms both the RPS and the Brier score, casting doubt on the value of non-locality and sensitivity to distance as properties of scoring rules in this context.

Keywords: football forecasting; forecast evaluation; ignorance score; ranked probability score; scoring rules (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 15 pages
Date: 2021-12-01
New Economics Papers: this item is included in nep-ban, nep-ecm, nep-for, nep-ore and nep-spo
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Published in Journal of Quantitative Analysis in Sports, 1, December, 2021, 17(4), pp. 273 - 287. ISSN: 1559-0410

Downloads: (external link)
http://eprints.lse.ac.uk/111494/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:111494

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().