Variable importance analysis with interpretable machine learning for fair risk prediction

Yilin Ning, Siqi Li, Yih Yng Ng, Michael Yih Chong Chia, Han Nee Gan, Ling Tiah, Desmond Renhao Mao, Wei Ming Ng, Benjamin Sieu-Hon Leong, Nausheen Doctor, Marcus Eng Hock Ong and Nan Liu

PLOS Digital Health, 2024, vol. 3, issue 7, 1-15

Abstract: Machine learning (ML) methods are increasingly used to assess variable importance, but such black box models lack stability when limited in sample sizes, and do not formally indicate non-important factors. The Shapley variable importance cloud (ShapleyVIC) addresses these limitations by assessing variable importance from an ensemble of regression models, which enhances robustness while maintaining interpretability, and estimates uncertainty of overall importance to formally test its significance. In a clinical study, ShapleyVIC reasonably identified important variables when the random forest and XGBoost failed to, and generally reproduced the findings from smaller subsamples (n = 2500 and 500) when statistical power of the logistic regression became attenuated. Moreover, ShapleyVIC reasonably estimated non-significant importance of race to justify its exclusion from the final prediction model, as opposed to the race-dependent model from the conventional stepwise model building. Hence, ShapleyVIC is robust and interpretable for variable importance assessment, with potential contribution to fairer clinical risk prediction.Author summary: Logistic regression analyses are often used to quantify variable importance to clinical outcomes, which provide data-driven evidence for subsequent clinical interventions. However, such analyses are often affected by sampling variability, especially when sample sizes are small. Machine learning (ML) methods have been explored for robust alternatives for variable importance assessments, but they are often not interpretable, and lack stability with small sample sizes. These limitations could be addressed by the Shapley variable importance cloud (ShapleyVIC), which enhances regression-based variable importance analysis using ML approaches, hence improving robustness while maintaining interpretability. Another desirable property of ShapleyVIC is that it estimates the uncertainty of overall variable importance to exclude variables with non-significant importance to the prediction. In a clinical study, ShapleyVIC reasonably identified important variables when the random forest and XGBoost had questionable findings. For smaller subsamples (n = 2500 and 500), the logistic regression identified fewer important variables, but ShapleyVIC generally reproduced the findings from the full cohort. Moreover, ShapleyVIC reasonably identified the low importance of race and excluded it from the final prediction model, but the conventional model building approach failed to. Hence, ShapleyVIC is robust and interpretable for variable importance assessment, with potential contribution to fairer clinical risk prediction.

Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000542 (text/html)
https://journals.plos.org/digitalhealth/article/fi ... 00542&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pdig00:0000542

DOI: 10.1371/journal.pdig.0000542

Access Statistics for this article

More articles in PLOS Digital Health from Public Library of Science
Bibliographic data for series maintained by digitalhealth ().