The impact of intransitivity on the Elo rating system

Adam H Hamilton, Anna Kalenkova and Matthew Roughan

PLOS ONE, 2025, vol. 20, issue 12, 1-25

Abstract: This paper studies the behaviour of the Elo rating system when the underlying modelling assumptions are not met. The Elo rating system is a popular statistical technique used to analyse pairwise comparison data. It is perhaps best known for rating chess players. The crucial assumption behind the Elo rating system is that the probability of which players wins a chess game depends primarily on a real-valued parameter, that quantifies the player’s “skill". This implicitly assumes that the binary relation “more likely to win against" is transitive. This paper studies how the Elo rating system behaves when this assumption is relaxed. First, we prove that once the assumption of transitivity is relaxed, the Elo ratings exhibits the undesirable property that estimated ratings are dependent on who plays who. Second, we prove that even when the assumption of transitivity is relaxed, for a given distribution with which players are selected, there is a unique point where the expected change of Elo ratings at that point is zero. This point represents the maximum likelihood estimator of the Elo ratings, given the observed data. Finally, we introduce a statistic that can be used to measure the intransitivity present in a game. We derive this measurement, and demonstrate on simulated data that it satisfies some useful sanity tests.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0338261 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 38261&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0338261

DOI: 10.1371/journal.pone.0338261

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().