Examination of Testlet Effect in Open-Ended Items

Gülden Kaya Uyanik and Levent Ertuna

SAGE Open, 2022, vol. 12, issue 1, 21582440221079849

Abstract: The testlets are prevalently used in national and international standard tests and the teacher-made achievement tests thanks to their practicality. The purpose of the current study is to evaluate the testlet effect in test consisting the open-ended items and to identify the possible errors when this effect is ignored. The data consists of the scores given by three raters of the responses of 380 students in the teacher training program to two open-ended items in two testlets. The two designs with and without the testlet effect were modeled (totally crossed random design and the nested random design) and the differences between them were analyzed by Generalizability Theory (GT). The results showed that raters had a slight effect on the scores in both models tested, and the reliability of the scores was overestimated when the testlet effect was not ignored in such measurement tools.

Keywords: generalizability theory; open-ended item; testlet; testlet effect; rater effect (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/21582440221079849 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:sagope:v:12:y:2022:i:1:p:21582440221079849

DOI: 10.1177/21582440221079849

Access Statistics for this article

More articles in SAGE Open
Bibliographic data for series maintained by SAGE Publications ().