Estimating multiple rater agreement for a rare diagnosis

Joseph S. Verducci, Michael E. Mack and Morris H. DeGroot

Journal of Multivariate Analysis, 1988, vol. 27, issue 2, 512-535

Abstract: This paper addresses the problem of estimating the population coefficient of agreement kappa (?) among a set of raters who independently classify a randomly selected subject into one of two categories. Of the many possible probability models for these classifications, only mixtures of binomial models incorporate random rater effects, although limiting forms of additive and multiplicative (log-linear) models may themselves be represented as mixtures of binomials. Mixture models also motivate a simple new estimator of ? that is appropriate in the important situation where one of the categories is rare. In the case of a rare category, simulations under multiplicative and mixture models demonstrate the substantially smaller mean squared error of compared to its more popular competitor. An example of psychiatric classification illustrates the plausibility of a simple mixture model as well as sizable discrepancies among estimators of ?.

Keywords: agreement; kappa; intraclass; correlation; reliability; log-linear; model; mixing; distribution (search for similar items in EconPapers)
Date: 1988
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/0047-259X(88)90145-5
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:jmvana:v:27:y:1988:i:2:p:512-535

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/supportfaq.cws_home/regional
https://shop.elsevie ... _01_ooc_1&version=01

Access Statistics for this article

Journal of Multivariate Analysis is currently edited by de Leeuw, J.

More articles in Journal of Multivariate Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().