Effects of Item Calibration Errors on Computerized Adaptive Testing under Cognitive Diagnosis Models

Hung-Yu Huang ()
Additional contact information
Hung-Yu Huang: University of Taipei

Journal of Classification, 2018, vol. 35, issue 3, No 4, 437-465

Abstract: Abstract In a cognitive diagnostic computerized adaptive testing (CD-CAT) exam, an item pool that consists of items with calibrated item parameters is used for item selection and attribute estimation. The parameter estimates for the items in the item pool are often treated as if they were the true population parameters, and therefore, the calibration errors are ignored. The purpose of this study was to investigate the effects of calibration errors on the attribute classification accuracy, the measurement precision of attribute mastery classification, and the test information under the log-linear cognitive diagnosis model (LCDM) framework. The deterministic input, noisy “and” gate (DINA) model and the compensatory re-parameterized unified model (C-RUM) were used in fixed-length CD-CAT simulations. The results showed that high levels of calibration errors were associated with low classification accuracy, low test information, and misleading estimation of measurement precision. The effects of calibration errors decreased as the test length increased, and the DINA model appeared to be more vulnerable in the presence of calibration errors. The C-RUM was less influenced by calibration errors because of its additive characteristics in the LCDM framework. The same conclusions applied when item exposure control was incorporated and when different item selection methods were used. Finally, the use of a larger calibration sample size to calibrate the item pool was found to reduce the magnitudes of error variances and increase the attribute classification accuracy.

Keywords: Computerized adaptive testing; Cognitive diagnosis models; Calibration errors; Log-linear cognitive diagnosis model (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00357-018-9265-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jclass:v:35:y:2018:i:3:d:10.1007_s00357-018-9265-y

Ordering information: This journal article can be ordered from
http://www.springer. ... hods/journal/357/PS2

DOI: 10.1007/s00357-018-9265-y

Access Statistics for this article

Journal of Classification is currently edited by Douglas Steinley

More articles in Journal of Classification from Springer, The Classification Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().