Detection and Treatment of Careless Responses to Improve Item Parameter Estimation

Jeffrey M. Patton, Ying Cheng, Maxwell Hong and Qi Diao
Additional contact information
Jeffrey M. Patton: Financial Industry Regulatory Authority (FINRA)
Maxwell Hong: University of Notre Dame
Qi Diao: Educational Testing Service

Journal of Educational and Behavioral Statistics, 2019, vol. 44, issue 3, 309-341

Abstract: In psychological and survey research, the prevalence and serious consequences of careless responses from unmotivated participants are well known. In this study, we propose to iteratively detect careless responders and cleanse the data by removing their responses. The careless responders are detected using person-fit statistics. In two simulation studies, the iterative procedure leads to nearly perfect power in detecting extremely careless responders and much higher power than the noniterative procedure in detecting moderately careless responders. Meanwhile, the false-positive error rate is close to the nominal level. In addition, item parameter estimation is much improved by iteratively cleansing the calibration sample. The bias in item discrimination and location parameter estimates is substantially reduced. The standard error estimates, which are spuriously small in the presence of careless responses, are corrected by the iterative cleansing procedure. An empirical example is also presented to illustrate the proposed procedure. These results suggest that the proposed procedure is a promising way to improve item parameter estimation for tests of 20 items or longer when data are contaminated by careless responses.

Keywords: careless responses; item response theory; person fit; sample cleansing; item parameter estimation (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998618825116 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:44:y:2019:i:3:p:309-341

DOI: 10.3102/1076998618825116

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().