Peirce's and Cohen's for Measures of Rater Reliability

Beau Abar and Eric Loken

Journal of Probability and Statistics, 2010, vol. 2010, 1-10

Abstract:

This study examined a historical mixture model approach to the evaluation of ratings made in “gold standard” and two-rater contingency tables. Peirce's and the derived average were discussed in relation to a widely used index of reliability in the behavioral sciences, Cohen's . Sample size, population base rate of occurrence, the true “science of the method”, and guessing rates were manipulated across simulations. In “gold standard” situations, Peirce's tended to recover the true reliability of ratings as well as better than . In two-rater situations, ave tended to recover the true reliability as well as better than in most situations. The empirical utility and potential theoretical benefits of mixture model methods in estimating reliability are discussed, as are the associations between the statistics and other modern mixture model approaches.

Date: 2010
References: Add references at CitEc
Citations:

Downloads: (external link)
http://downloads.hindawi.com/journals/JPS/2010/480364.pdf (application/pdf)
http://downloads.hindawi.com/journals/JPS/2010/480364.xml (text/xml)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hin:jnljps:480364

DOI: 10.1155/2010/480364

Access Statistics for this article

More articles in Journal of Probability and Statistics from Hindawi
Bibliographic data for series maintained by Mohamed Abdelhakeem ().