Nonparametric Limits of Agreement for Small to Moderate Sample Sizes: A Simulation Study

Maria E. Frey, Hans C. Petersen and Oke Gerke
Additional contact information
Maria E. Frey: Department of Toxicology, Charles River Laboratories Copenhagen A/S, 4623 Lille Skensved, Denmark
Hans C. Petersen: Department of Mathematics and Computer Science, University of Southern Denmark, 5230 Odense M, Denmark
Oke Gerke: Department of Nuclear Medicine, Odense University Hospital, 5000 Odense C, Denmark

Stats, 2020, vol. 3, issue 3, 1-13

Abstract: The assessment of agreement in method comparison and observer variability analysis of quantitative measurements is usually done by the Bland–Altman Limits of Agreement, where the paired differences are implicitly assumed to follow a normal distribution. Whenever this assumption does not hold, the 2.5% and 97.5% percentiles are obtained by quantile estimation. In the literature, empirical quantiles have been used for this purpose. In this simulation study, we applied both sample, subsampling, and kernel quantile estimators, as well as other methods for quantile estimation to sample sizes between 30 and 150 and different distributions of the paired differences. The performance of 15 estimators in generating prediction intervals was measured by their respective coverage probability for one newly generated observation. Our results indicated that sample quantile estimators based on one or two order statistics outperformed all of the other estimators and they can be used for deriving nonparametric Limits of Agreement. For sample sizes exceeding 80 observations, more advanced quantile estimators, such as the Harrell–Davis and estimators of Sfakianakis–Verginis type, which use all of the observed differences, performed likewise well, but may be considered intuitively more appealing than simple sample quantile estimators that are based on only two observations per quantile.

Keywords: agreement; Bland-Altman plot; coverage; limits of agreement; method comparison; quantile estimation; repeatability; reproducibility (search for similar items in EconPapers)
JEL-codes: C1 C10 C11 C14 C15 C16 (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2571-905X/3/3/22/pdf (application/pdf)
https://www.mdpi.com/2571-905X/3/3/22/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jstats:v:3:y:2020:i:3:p:22-355:d:405494

Access Statistics for this article

Stats is currently edited by Mrs. Minnie Li

More articles in Stats from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().