Generalizing Beyond the Test: Permutation-Based Profile Analysis for Explaining DIF Using Item Features

Maria Bolsinova, Jesper Tijmstra, Leslie Rutkowski and David Rutkowski
Additional contact information
Jesper Tijmstra: Tilburg University
David Rutkowski: Indiana University Centre for Educational Measurement

Journal of Educational and Behavioral Statistics, 2024, vol. 49, issue 2, 207-240

Abstract: Profile analysis is one of the main tools for studying whether differential item functioning can be related to specific features of test items. While relevant, profile analysis in its current form has two restrictions that limit its usefulness in practice: It assumes that all test items have equal discrimination parameters, and it does not test whether conclusions about the item-feature effects generalize outside of the considered set of items. This article addresses both of these limitations, by generalizing profile analysis to work under the two-parameter logistic model and by proposing a permutation test that allows for generalizable conclusions about item-feature effects. The developed methods are evaluated in a simulation study and illustrated using Programme for International Student Assessment 2015 Science data.

Keywords: differential item functioning; generalizability; item features; international large-scale assessment; profile analysis (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/10769986231174927 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:49:y:2024:i:2:p:207-240

DOI: 10.3102/10769986231174927

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().