Receiver Operating Characteristic Prediction for Classification: Performances in Cross-Validation by Example

Andra Ciocan, Nadim Al Hajjar, Florin Graur, Valentin C. Oprea, Răzvan A. Ciocan and Sorana D. Bolboacă
Additional contact information
Andra Ciocan: Department of Medical Informatics and Biostatistics, “Iuliu Hațieganu” University of Medicine and Pharmacy Cluj-Napoca, Louis Pasteur Street, No. 6, 400349 Cluj-Napoca, Romania
Nadim Al Hajjar: “Prof. Dr. Octavian Fodor” Regional Institute of Gastroenterology and Hepatology Cluj-Napoca, Croitorilor Street, No. 19–21, 400162 Cluj-Napoca, Romania
Florin Graur: “Prof. Dr. Octavian Fodor” Regional Institute of Gastroenterology and Hepatology Cluj-Napoca, Croitorilor Street, No. 19–21, 400162 Cluj-Napoca, Romania
Valentin C. Oprea: “Dr. Constantin Papilian” Military Emergency Hospital Cluj-Napoca, General Traian Moșoiu Street, No. 22, 400132 Cluj-Napoca, Romania
Răzvan A. Ciocan: Department of Medical Skills—Human Sciences, “Iuliu Hațieganu” University of Medicine and Pharmacy Cluj-Napoca, Marinescu Street, No. 23, 400337 Cluj-Napoca, Romania
Sorana D. Bolboacă: Department of Medical Informatics and Biostatistics, “Iuliu Hațieganu” University of Medicine and Pharmacy Cluj-Napoca, Louis Pasteur Street, No. 6, 400349 Cluj-Napoca, Romania

Mathematics, 2020, vol. 8, issue 10, 1-17

Abstract: The stability of receiver operating characteristic in context of random split used in development and validation sets, as compared to the full models for three inflammatory ratios (neutrophil-to-lymphocyte (NLR), derived neutrophil-to-lymphocyte (dNLR) and platelet-to-lymphocyte (PLR) ratio) evaluated as predictors for metastasis in patients with colorectal cancer, was investigated. Data belonging to patients admitted with the diagnosis of colorectal cancer from January 2014 until September 2019 in a single hospital were used. There were 1688 patients eligible for the study, 418 in the metastatic stage. All investigated inflammatory ratios proved to be significant classification models on both the full models and on cross-validations (AUCs > 0.05). High variability of the cut-off values was observed in the unrestricted and restricted split (full models: 4.255 for NLR, 2.745 for dNLR and 255.56 for PLR; random splits: cut-off from 3.215 to 5.905 for NLR, from 2.625 to 3.575 for dNLR and from 134.67 to 335.9 for PLR), but with no effect on the models characteristics or performances. The investigated biomarkes proved limited value as predictors for metastasis (AUCs < 0.8), with largely sensitivity and specificity (from 33.3% to 79.2% for the full model and 29.1% to 82.7% in the restricted splits). Our results showed that a simple random split of observations, weighting or not the patients with and whithout metastasis, in a ROC analysis assures the performances similar to the full model, if at least 70% of the available population is included in the study.

Keywords: receiver operating characteristic; area under the curve; models performances; biomarkers; metastasis (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2227-7390/8/10/1741/pdf (application/pdf)
https://www.mdpi.com/2227-7390/8/10/1741/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:8:y:2020:i:10:p:1741-:d:425988

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().