Detecting Noneffortful Responses Based on a Residual Method Using an Iterative Purification Process

Yue Liu and Hongyun Liu
Additional contact information
Yue Liu: Institute of Brain and Psychological Sciences, Sichuan Normal University
Hongyun Liu: Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University

Journal of Educational and Behavioral Statistics, 2021, vol. 46, issue 6, 717-752

Abstract: The prevalence and serious consequences of noneffortful responses from unmotivated examinees are well-known in educational measurement. In this study, we propose to apply an iterative purification process based on a response time residual method with fixed item parameter estimates to detect noneffortful responses. The proposed method is compared with the traditional residual method and noniterative method with fixed item parameters in two simulation studies in terms of noneffort detection accuracy and parameter recovery. The results show that when severity of noneffort is high, the proposed method leads to a much higher true positive rate with a small increase of false discovery rate. In addition, parameter estimation is significantly improved by the strategies of fixing item parameters and iteratively cleansing. These results suggest that the proposed method is a potential solution to reduce the impact of data contamination due to severe low test-taking effort and to obtain more accurate parameter estimates. An empirical study is also conducted to show the differences in the detection rate and parameter estimates among different approaches.

Keywords: noneffortful response; response time; residual method; mixture model; iterative purification (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998621994366 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:46:y:2021:i:6:p:717-752

DOI: 10.3102/1076998621994366

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().