Confidence Intervals for Intraclass Correlation in Inter‐Rater Reliability

Valentin Rousson, Theo Gasser and Burkhardt Seifert

Scandinavian Journal of Statistics, 2003, vol. 30, issue 3, 617-624

Abstract: Abstract Calculation of a confidence interval for intraclass correlation to assess inter‐rater reliability is problematic when the number of raters is small and the rater effect is not negligible. Intervals produced by existing methods are uninformative: the lower bound is often close to zero, even in cases where the reliability is good and the sample size is large. In this paper, we show that this problem is unavoidable without extra assumptions and we propose two new approaches. The first approach assumes that the raters are sufficiently trained and is related to a sensitivity analysis. The second approach is based on a model with fixed rater effect. Using either approach, we obtain conservative and informative confidence intervals even from samples with only two raters. We illustrate our point with data on the development of neuromotor functions in children and adolescents.

Date: 2003
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/1467-9469.00353

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:scjsta:v:30:y:2003:i:3:p:617-624

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0303-6898

Access Statistics for this article

Scandinavian Journal of Statistics is currently edited by ÿrnulf Borgan and Bo Lindqvist

More articles in Scandinavian Journal of Statistics from Danish Society for Theoretical Statistics, Finnish Statistical Society, Norwegian Statistical Association, Swedish Statistical Association
Bibliographic data for series maintained by Wiley Content Delivery ().