Judging the judges: evaluating the accuracy and national bias of international gymnastics judges

Heiniger Sandro () and Mercier Hugues ()
Additional contact information
Heiniger Sandro: Swiss Institute for Empirical Economic Research, University of St.Gallen, St.Gallen, Switzerland
Mercier Hugues: University of Applied Sciences and Arts of Western Switzerland, Neuchâtel, Switzerland

Journal of Quantitative Analysis in Sports, 2021, vol. 17, issue 4, 289-305

Abstract: We design, describe and implement a statistical engine to analyze the performance of gymnastics judges with three objectives: (1) provide constructive feedback to judges, executive committees and national federations; (2) assign the best judges to the most important competitions; (3) detect bias and persistent misjudging. Judging a gymnastics routine is a random process, and we model this process using heteroscedastic random variables. The developed marking score scales the difference between the mark of a judge and the true performance level of a gymnast as a function of the intrinsic judging error variability estimated from historical data for each apparatus. This dependence between judging variability and performance quality has never been properly studied. We leverage the intrinsic judging error variability and the marking score to detect outlier marks and study the national bias of judges favoring athletes of the same nationality. We also study ranking scores assessing to what extent judges rate gymnasts in the correct order. Our main observation is that there are significant differences between the best and worst judges, both in terms of accuracy and national bias. The insights from this work have led to recommendations and rule changes at the Fédération Internationale de Gymnastique.

Keywords: gymnastics; intrinsic judging error variability; national bias; outlier detection; quantifying accuracy; sports judges (search for similar items in EconPapers)
Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1515/jqas-2019-0113 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:jqsprt:v:17:y:2021:i:4:p:289-305:n:3

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/jqas/html

DOI: 10.1515/jqas-2019-0113

Access Statistics for this article

Journal of Quantitative Analysis in Sports is currently edited by Mark Glickman

More articles in Journal of Quantitative Analysis in Sports from De Gruyter
Bibliographic data for series maintained by Peter Golla ().