Taking PISA Seriously: How Accurate are Low-Stakes Exams?

Pelin Akyol, Kala Krishna () and Jinwen Wang ()
Additional contact information
Jinwen Wang: Bates White Economic Consulting

Journal of Labor Research, 2021, vol. 42, issue 2, No 3, 184-243

Abstract: Abstract PISA is seen as the gold standard for evaluating educational outcomes worldwide. Yet, being a low-stakes exam, students may not take it seriously resulting in downward biased scores and inaccurate rankings. This paper provides a method to identify and account for non-serious behavior in low-stakes exams by leveraging information in computer-based assessments in PISA 2015. Our method corrects for non-serious behavior by fully imputing scores for items not taken seriously. We compare the scores/rankings calculated by our method to the scores/rankings calculated by giving zero points to skipped items as well as to the scores/rankings calculated by treating skipped items at the end of the exam as if they were not administered, which is the procedure followed by PISA. We show that a country can improve its ranking by up to 15 places by encouraging its own students to take the exam seriously and that the PISA approach corrects for only about half of the bias generated by the non-seriousness.

Keywords: Low-stakes exams; Computer-based assessments; PISA; Biased rankings; Item response data (search for similar items in EconPapers)
JEL-codes: C53 I20 I21 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (14)

Downloads: (external link)
http://link.springer.com/10.1007/s12122-021-09317-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
Working Paper: Taking PISA Seriously: How Accurate are Low Stakes Exams? (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jlabre:v:42:y:2021:i:2:d:10.1007_s12122-021-09317-8

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/12122

DOI: 10.1007/s12122-021-09317-8

Access Statistics for this article

Journal of Labor Research is currently edited by Ozkan Eren

More articles in Journal of Labor Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().