Cohen’s Kappa Coefficient as a Measure to Assess Classification Improvement following the Addition of a New Marker to a Regression Model

Barbara Więckowska (), Katarzyna B. Kubiak, Paulina Jóźwiak, Wacław Moryson and Barbara Stawińska-Witoszyńska
Additional contact information
Barbara Więckowska: Department of Computer Science and Statistics, Poznan University of Medical Sciences, 60-806 Poznan, Poland
Katarzyna B. Kubiak: Department of Computer Science and Statistics, Poznan University of Medical Sciences, 60-806 Poznan, Poland
Paulina Jóźwiak: Department of Preventive Medicine, Poznan University of Medical Sciences, 60-781 Poznan, Poland
Wacław Moryson: Department of Epidemiology and Hygiene, Chair of Social Medicine, Poznan University of Medical Sciences, 60-806 Poznan, Poland
Barbara Stawińska-Witoszyńska: Department of Epidemiology and Hygiene, Chair of Social Medicine, Poznan University of Medical Sciences, 60-806 Poznan, Poland

IJERPH, 2022, vol. 19, issue 16, 1-15

Abstract: The need to search for new measures describing the classification of a logistic regression model stems from the difficulty in searching for previously unknown factors that predict the occurrence of a disease. A classification quality assessment can be performed by testing the change in the area under the receiver operating characteristic curve (AUC). Another approach is to use the Net Reclassification Improvement (NRI), which is based on a comparison between the predicted risk, determined on the basis of the basic model, and the predicted risk that comes from the model enriched with an additional factor. In this paper, we draw attention to Cohen’s Kappa coefficient, which examines the actual agreement in the correction of a random agreement. We proposed to extend this coefficient so that it may be used to detect the quality of a logistic regression model reclassification. The results provided by Kappa‘s reclassification were compared with the results obtained using NRI. The random variables’ distribution attached to the model on the classification change, measured by NRI, Kappa, and AUC, was presented. A simulation study was conducted on the basis of a cohort containing 3971 Poles obtained during the implementation of a lower limb atherosclerosis prevention program.

Keywords: Cohen’s Kappa coefficient; NRI net reclassification ratio; logistic regression model; reclassification; AUC (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/19/16/10213/pdf (application/pdf)
https://www.mdpi.com/1660-4601/19/16/10213/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:19:y:2022:i:16:p:10213-:d:890547

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().