Accounting for measurement bias: a new framework for reliable country ranking in large-scale educational assessments

Jing Ouyang, Yunxiao Chen, Chengcheng Li and Gongjun Xu

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: International Large-scale Assessments (ILSAs), such as the Program for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), are cornerstone tools for global educational research and policy-making. By benchmarking educational quality and performance trends, these assessments enable countries to evaluate and share effective pedagogical structures. Specifically, ILSAs employ Item Response Theory (IRT) models to rank countries by students’ performance on cognitive items. However, measurement bias—arising from linguistic, cultural, and curricular differences—poses a significant threat to the statistical inference of IRT models and, consequently, the validity of the resulting rankings. Neglecting this bias can lead to systematic errors in parameter estimation, ultimately distorting national standings. To address this, we propose a novel method that avoids the restrictive assumptions typical of existing approaches, such as the prior identification of unbiased “anchor items” or designated reference groups. Our approach is computationally efficient and provides theoretical guarantees for the reliable recovery of group rankings. We apply this method to PISA 2022 data across the mathematics, science, and reading domains, yielding corrected performance rankings and insights into the survey’s measurement-bias structures. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Keywords: latent variable models; psychometrics; differential item functioning; measurement invariance; item response theory (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 23 pages
Date: 2026-06-29
New Economics Papers: this item is included in nep-mac
References: Add references at CitEc
Citations:

Published in Journal of the American Statistical Association, 29, June, 2026. ISSN: 0162-1459

Downloads: (external link)
https://researchonline.lse.ac.uk/id/eprint/138283/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:138283

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().