Estimating Classification Accuracy for Complex Decision Rules Based on Multiple Scores

Karen M. Douglas and Robert J. Mislevy

Journal of Educational and Behavioral Statistics, 2010, vol. 35, issue 3, 280-306

Abstract: Important decisions about students are made by combining multiple measures using complex decision rules. Although methods for characterizing the accuracy of decisions based on a single measure have been suggested by numerous researchers, such methods are not useful for estimating the accuracy of decisions based on multiple measures. This study presents a simulation method useful for estimating classification accuracy for any measurement model and complex decision rule. The utility of the method is demonstrated by application to actual data from General Educational Development (GED) test takers. Findings in this study illustrate how multiple scores that are combined can have a large impact on the accuracy of the resulting decision. Furthermore, choice of measure of agreement is of central importance in describing decision accuracy.

Keywords: decision making; high-stakes testing; policy; testing; validity/reliability (search for similar items in EconPapers)
Date: 2010
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998609346969 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:35:y:2010:i:3:p:280-306

DOI: 10.3102/1076998609346969

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().