Understanding Performance in Test Taking: The Role of Question Difficulty Order

Nagore Iriberri, Lina Anaya, Pedro Rey Biel and Gema Zamarro
Authors registered in the RePEc Author Service: Pedro Rey-Biel

No 16099, CEPR Discussion Papers from C.E.P.R. Discussion Papers

Abstract: Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty and we find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers. Consistent results are found exploiting the random variation of difficulty across test booklets in the Programme for International Student Assessment (PISA), a triannual international test, for the years of 2009, 2012, and 2015, providing additional external validity. We conclude that the order of the difficulty of the questions in tests should be considered carefully, in particular when comparing performance between test-takers who have faced different order of questions

JEL-codes: C93 D81 I20 J16 (search for similar items in EconPapers)
Date: 2021-05
References: Add references at CitEc
Citations:

Downloads: (external link)
https://cepr.org/publications/DP16099 (application/pdf)
CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org

Related works:
Journal Article: Understanding performance in test taking: The role of question difficulty order (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cpr:ceprdp:16099

Ordering information: This working paper can be ordered from
https://cepr.org/publications/DP16099

Access Statistics for this paper

More papers in CEPR Discussion Papers from C.E.P.R. Discussion Papers Centre for Economic Policy Research, 33 Great Sutton Street, London EC1V 0DX.
Bibliographic data for series maintained by ().