Item characteristics and test-taker disengagement in PISA

Francesco Avvisati, Janine Buchholz, Mario Piacentini and Luis Francisco Vargas Madriz

No 312, OECD Education Working Papers from OECD Publishing

Abstract: If test-takers do not engage with the assessment, the reliability of test scores and the validity of inferences about their proficiency may suffer. Test-taker disengagement is particularly likely in low-stakes assessments and, according to prior research, for certain types of students. But levels of engagement may also be related to aspects that test developers can manipulate, such as item characteristics. This paper investigates which item characteristics are associated with two indicators of test-taker disengagement, rapid guessing and breakoffs, in an international assessment of reading. Analyses of data from almost 500 000 students from 67 countries and economies that took part in the 2018 Programme for International Student Assessment (PISA) show that rapid guessing was observed mainly on simple multiple-choice questions. Breakoffs were more likely in the presence of idiosyncratic selected-response formats, such as hot spot or matching tasks. Both rapid guessing and breakoffs were more frequent on tasks involving long and complex texts.

Date: 2024-04-12
New Economics Papers: this item is included in nep-ure
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1787/7abea67b-en (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oec:eduaab:312-en

Access Statistics for this paper

More papers in OECD Education Working Papers from OECD Publishing Contact information at EDIRC.
Bibliographic data for series maintained by ().