Improved Type I Error Control and Reduced Estimation Bias for DIF Detection Using SIBTEST

Hai Jiang and William Stout

Journal of Educational and Behavioral Statistics, 1998, vol. 23, issue 4, 291-322

Abstract: One emphasis in the development and evaluation of SIBTEST has been the control of Type I error (false flagging of non-differential item functioning [DIF] items) inflation and estimation bias. SIBTEST has performed well in comparative simulation studies of Type I error and estimation bias relative to other procedures such as the Mantel-Haenszel and Logistic Regression. Nevertheless it has for a minority of cases that might occur in applications displayed sizable Type I error inflation and estimation bias. A vital part of SIBTEST is the regression correction, which adjusts for the Type I error-inflating and estimation-biasing influence of group target ability differences by using the linear regression of true on observed matching subtest scores from Classical Test Theory. In this paper, we propose a new regression correction, using essentially a two-segment piecewise linear regression of the true on observed matching subtest scores. A realistic simulation study of the new approach shows that when there is a clear group ability distributional difference, the new approach displays improved SIBTEST Type I error performance; when there is no group ability distributional difference, its Type I error rate is comparable to the current SIBTEST. We have also conducted a power study which indicates that the new approach has on average similar power as the current SIBTEST. We concluded that the new version of SIBTEST, although not perfectly robust, seems appropriately robust against sizable Type I error inflation, while retaining other desirable features of the current version.

Keywords: Keywords: differential item functioning; DIF; item response function; IRF; Mantel-Haenszel; regression correction; SIBTEST (search for similar items in EconPapers)
Date: 1998
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/10769986023004291 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:23:y:1998:i:4:p:291-322

DOI: 10.3102/10769986023004291

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().