Measuring Consensus in Binary Forecasts: NFL Game Predictions

ChiUng Song (), Bryan Boulier and Herman Stekler
Additional contact information
ChiUng Song: Science and Technology Policy Institute

No 2008-006, Working Papers from The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting

Abstract: Previous research on defining and measuring consensus (agreement) among forecasters has been concerned with evaluation of forecasts of continuous variables. This previous work is not relevant when the forecasts involve binary decisions: up-down or win-lose. In this paper we use Cohen¡¯s kappa coefficient, a measure of inter-rater agreement involving binary choices, to evaluate forecasts of National Football League games. This statistic is applied to the forecasts of 74 experts and 31 statistical systems that predicted the outcomes of games during two NFL seasons. We conclude that the forecasters, particularly the systems, displayed significant levels of agreement and that levels of agreement in picking game winners were higher than in picking against the betting line. There is greater agreement among statistical systems in picking game winners or picking winners against the line as the season progresses, but no change in levels of agreement among experts. High levels of consensus among forecasters are associated with greater accuracy in picking game winners, but not in picking against the line.

Keywords: binary forecasts; NFL; agreement; consensus; kappa coefficient (search for similar items in EconPapers)
Pages: 27 pages
Date: 2008-07
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www2.gwu.edu/~forcpgm/2008-006.pdf First version, 2008 (application/pdf)

Related works:
Journal Article: Measuring consensus in binary forecasts: NFL game predictions (2009)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gwc:wpaper:2008-006

Access Statistics for this paper

More papers in Working Papers from The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting Contact information at EDIRC.
Bibliographic data for series maintained by GW Economics Department ().