The Analysis of Multiple Choice Tests in Educational Assessment

Simon French
Additional contact information
Simon French: University of Manchester, Statistical Laboratory Department of Mathematics

A chapter in Probability and Bayesian Statistics, 1987, pp 175-182 from Springer

Abstract: Abstract Multiple choice tests are much, but not exclusively, used in the British public examinations system. The analysis of results from such tests has been subject to much debate, particularly concerning the appropriateness of latent trait models. In this paper I adopt an entirely subjectivist approach. I believe the purpose of a public examination is not to measure in some objective sense the performances of candidates, but rather to report the judgements of examiners as to those performances. It is the examiners’ judgements that are modelled by marks and grades, not something directly about the candidates themselves. Adopting this viewpoint, I make two groups of comments pertinent to multiple choice tests. First, if one is to use latent trait models to analyse candidate responses, then one must be clear as to the meaning of parameters within the models. I argue that latent trait variables are technical devices which encode certain expectations about the data, but other than that they have no physical meaning. Because of this view, I shall argue that latent trait models are appropriate for critically evaluating assumptions about examination data, but are inappropriate for the purpose of ranking candidates’ work to report and grade individual performances. Second, one should consider in what form to elicit responses from the candidates. De Finetti suggested that candidates should respond with their probability of the correctness of each possible answer to an item and that these responses should be assessed by means of a scoring rule. However, such schemes have many problems: the difficulty of getting candidates, still at school, to accept the inevitability of uncertainty in their lives; the problem of calibration, because they are unlikely to be equally good probability assessors. Perhaps more serious is the difficulty that a scoring rule which encourages a candidate to honestly reveal his beliefs may not reflect the manner in which the examiners wish to judge the candidate.

Keywords: Latent Trait; Multiple Choice Test; Public Examination; Subjectivist Approach; Objective Sense (search for similar items in EconPapers)
Date: 1987
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4613-1885-9_18

Ordering information: This item can be ordered from
http://www.springer.com/9781461318859

DOI: 10.1007/978-1-4613-1885-9_18

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().