DIF Analysis with Multilevel Data: A Simulation Study Using the Latent Variable Approach

Ying Jin and Hershel Eason

Journal of Educational Issues, 2016, vol. 2, issue 2, 290306

Abstract: The effects of mean ability difference (MAD) and short tests on the performance of various DIF methods have been studied extensively in previous simulation studies. Their effects, however, have not been studied under multilevel data structure. MAD was frequently observed in large-scale cross-country comparison studies where the primary sampling units were more likely to be clusters (e.g., schools). With short tests, regular DIF methods under MAD-present conditions might suffer from inflated type I error rate due to low reliability of test scores, which would adversely impact the matching ability of the covariate (i.e., the total score) in DIF analysis. The current study compared the performance of three DIF methods- logistic regression (LR), hierarchical logistic regression (HLR) taking multilevel structure into account, and hierarchical logistic regression with latent covariate (HLR-LC) taking multilevel structure into account as well as accounting for low reliability and MAD. The results indicated that HLR-LC outperformed both LR and HLR under most simulated conditions, especially under the MAD-present conditions when tests were short. Practical implications of the implementation of HLR-LC were also discussed.

Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.macrothink.org/journal/index.php/jei/article/download/10045/8358 (application/pdf)
https://www.macrothink.org/journal/index.php/jei/article/view/10045 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mth:jeijnl:v:2:y:2016:i:2:p:290306

Access Statistics for this article

Journal of Educational Issues is currently edited by Priscilla Ann

More articles in Journal of Educational Issues from Macrothink Institute
Bibliographic data for series maintained by Technical Support Office ().