Performance of tests based on the area under the ROC curve for multireader diagnostic data

Yi-Ting Hwang, Ya-Ru Hsu and Nan-Cheng Su

Journal of Applied Statistics, 2025, vol. 52, issue 3, 555-577

Abstract: One of the main objectives of disease prevention is to lower the healthcare costs and improve the quality of life. To achieve this, reliable diagnostic tools are needed. The diagnostic performance of a tool can be measured by the ROC curve and the AUC. However, some diagnostic tools such as MRI images are not objective, but depend on the interpretation of experts. Therefore, the accuracy of these tools may vary depending on who is interpreting them. To account for possible correlations when multiple readers collect data, Dorfman, Berbaum and Metz (1992) proposed using AUC pseudovalues from the jackknife sampling method and applying them to the mixed model to analyze the diagnostic reagent's accuracy. However, pseudovalues may go beyond the AUC range. Also, the random effect estimate may be negative due to a small number of readers. This paper develops tests based on AUC estimates and gives their asymptotic distribution. Moreover, a two-stage test is suggested to correct for negative random effect estimates. Four tests are created in total and their performance is evaluated by Monte Carlo simulations. The distributional assumption's robustness of these tests is checked, and their applicability is demonstrated by two real data sets.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/02664763.2024.2374931 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:japsta:v:52:y:2025:i:3:p:555-577

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/CJAS20

DOI: 10.1080/02664763.2024.2374931

Access Statistics for this article

Journal of Applied Statistics is currently edited by Robert Aykroyd

More articles in Journal of Applied Statistics from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().