Measuring consensus in binary forecasts: NFL game predictionsChiUng Song, Bryan Leslie Boulier and Herman O. Stekler () International Journal of Forecasting, 2009, vol. 25, issue 1, pages 182-191 Abstract: Previous research on defining and measuring consensus (agreement) among forecasters has been concerned with the evaluation of forecasts of continuous variables. This previous work is not relevant when the forecasts involve binary decisions: up-down or win-lose. In this paper we use Cohen's kappa coefficient, a measure of inter-rater agreement involving binary choices, to evaluate forecasts of National Football League games. This statistic is applied to the forecasts of 74 experts and 31 statistical systems that predicted the outcomes of games during two NFL seasons. We conclude that the forecasters, particularly the systems, displayed significant levels of agreement, and that levels of agreement in picking game winners were higher than in picking against the betting line. There is greater agreement among statistical systems in picking game winners or picking winners against the line as the season progresses, but no change in levels of agreement among experts. Higher levels of consensus among forecasters are associated with greater accuracy in picking game winners, but not in picking against the line. Keywords: Binary; forecasts; NFL; Agreement; Consensus; Kappa; coefficient (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: http://EconPapers.repec.org/RePEc:eee:intfor:v:25:y:2009:i:1:p:182-191 Access Statistics for this article International Journal of Forecasting is edited by R. J. Hyndman More articles in International Journal of Forecasting from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
Swedish Business School at Örebro University.