Understanding performance in test taking: The role of question difficulty order

Lina Anaya, Nagore Iriberri, Pedro Rey-Biel and Gema Zamarro

Economics of Education Review, 2022, vol. 90, issue C

Abstract: Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life and to evaluate and compare educational systems. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty and we find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers. We obtain similar results when exploiting the random variation of difficulty across test booklets in the Programme for International Student Assessment (PISA) for the years 2009, 2012, and 2015, which provides additional external validity to the experiment. We conclude that question difficulty order in tests has important policy implications for optimal test design and performance. It additionally may have important implications for ranking candidates, as well as for the evaluation and comparison of educational institutions and systems.

Keywords: Question order; Difficulty; Test performance (search for similar items in EconPapers)
JEL-codes: C93 D81 I20 J16 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S027277572200067X
Full text for ScienceDirect subscribers only

Related works:
Working Paper: Understanding Performance in Test Taking: The Role of Question Difficulty Order (2021)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ecoedu:v:90:y:2022:i:c:s027277572200067x

DOI: 10.1016/j.econedurev.2022.102293

Access Statistics for this article

Economics of Education Review is currently edited by E. Cohn

More articles in Economics of Education Review from Elsevier
Bibliographic data for series maintained by Catherine Liu ().