A Simulation Study of Reliability and Validity of Multiple-Choice Test Scores Under Six Response-Scoring Modes

Robert B. Frary

Journal of Educational and Behavioral Statistics, 1982, vol. 7, issue 4, 333-351

Abstract: Responses to a 40-item, four-choice test were simulated for 120 examinees under six response-scoring modes including number-right, corrected-for-guessing and answer-until-correct. Separate score sets were generated to reflect five levels of prevalence of misinformation (belief that an answer is a distractor) and five levels of propensity-to-guess contrary to instructions for modes designed to inhibit guessing. Criteria were simulated using the number-right mode with five levels of misinformation prevalence and four levels of true-score relationship with the predictor. The entire process was repeated with the introduction of normally distributed, random error at the item level. This process yielded 260 sets of five scores (predictor and four criteria), which were examined to determine differential effects on reliability and validity attributable to the response-scoring modes. Modes permitting multiple responses to an item were found to yield genuine increases in internal consistency reliability, which tended to carry over into validity coefficients. However, the validity differences among all the response-scoring modes simulated were small, probably too small to justify the additional cost and complexity of modes other than number-right.

Keywords: guessing; misinformation; multiple-choice; partial information; reliability; validity (search for similar items in EconPapers)
Date: 1982
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/10769986007004333 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:7:y:1982:i:4:p:333-351

DOI: 10.3102/10769986007004333

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().