Machine-learning-based prediction of disability progression in multiple sclerosis: An observational, international, multi-center study

Edward De Brouwer, Thijs Becker, Lorin Werthen-Brabants, Pieter Dewulf, Dimitrios Iliadis, Cathérine Dekeyser, Guy Laureys, Bart Van Wijmeersch, Veronica Popescu, Tom Dhaene, Dirk Deschrijver, Willem Waegeman, Bernard De Baets, Michiel Stock, Dana Horakova, Francesco Patti, Guillermo Izquierdo, Sara Eichau, Marc Girard, Alexandre Prat, Alessandra Lugaresi, Pierre Grammond, Tomas Kalincik, Raed Alroughani, Francois Grand’Maison, Olga Skibina, Murat Terzi, Jeannette Lechner-Scott, Oliver Gerlach, Samia J Khoury, Elisabetta Cartechini, Vincent Van Pesch, Maria José Sà, Bianca Weinstock-Guttman, Yolanda Blanco, Radek Ampapa, Daniele Spitaleri, Claudio Solaro, Davide Maimone, Aysun Soysal, Gerardo Iuliano, Riadh Gouider, Tamara Castillo-Triviño, José Luis Sánchez-Menoyo, Guy Laureys, Anneke van der Walt, Jiwon Oh, Eduardo Aguera-Morales, Ayse Altintas, Abdullah Al-Asmi, Koen de Gans, Yara Fragoso, Tunde Csepany, Suzanne Hodgkinson, Norma Deri, Talal Al-Harbi, Bruce Taylor, Orla Gray, Patrice Lalive, Csilla Rozsa, Chris McGuigan, Allan Kermode, Angel Pérez Sempere, Simu Mihaela, Magdolna Simo, Todd Hardy, Danny Decoo, Stella Hughes, Nikolaos Grigoriadis, Attila Sas, Norbert Vella, Yves Moreau and Liesbet Peeters

PLOS Digital Health, 2024, vol. 3, issue 7, 1-25

Abstract: Background: Disability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of the probability of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking. Methods: Data of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were followed. Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expanded disability status scale, treatment, relapse information, and MS course. To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated with the area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error. All our preprocessing and model code are available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS. Findings: Machine learning models achieved a ROC-AUC of 0⋅71 ± 0⋅01, an AUC-PR of 0⋅26 ± 0⋅02, a Brier score of 0⋅1 ± 0⋅01 and an expected calibration error of 0⋅07 ± 0⋅04. The history of disability progression was identified as being more predictive for future disability progression than the treatment or relapses history. Conclusions: Good discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This suggests machine-learning models can reliably inform clinicians about the future occurrence of progression and are mature for a clinical impact study. Author summary: Models that accurately predict disability progression in individuals with multiple sclerosis (MS) have the potential to greatly benefit both patients and medical professionals. By aiding in life planning and treatment decision-making, these predictive models can enhance the overall quality of care for people with MS. While previous academic literature has demonstrated the feasibility of predicting disability progression, recent systematic reviews have shed light on several methodological limitations within the existing research. These reviews have highlighted concerns such as the absence of probability calibration assessment, potential biases in cohort selection, and insufficient external validation. Furthermore, the datasets examined often include variables that are not routinely collected in clinical settings or readily available for digital analysis. Consequently, it remains uncertain whether the models identified in these systematic reviews can be effectively implemented in a clinical context. Compounding this issue, the lack of availability of data and analysis code makes it challenging to compare results across different publications. To address these gaps, this study endeavors to develop and validate a machine-learning-based prediction model using the largest longitudinal patient cohort ever assembled for disability progression prediction in MS. Leveraging data from MSBase, a comprehensive international data registry comprising information from multiple MS centers, we aim to create robust models capable of accurately predicting the probability of disability progression. The integration of machine learning models into routine clinical practice has the potential to greatly enhance treatment decision-making and life planning for individuals with MS. The models developed through this study could be subsequently evaluated in a clinical impact study involving MS centers participating in MSBase. This research represents a significant advancement towards the practical application of machine learning models in improving the treatment and care of individuals with MS.

Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000533 (text/html)
https://journals.plos.org/digitalhealth/article/fi ... 00533&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pdig00:0000533

DOI: 10.1371/journal.pdig.0000533

Access Statistics for this article

More articles in PLOS Digital Health from Public Library of Science
Bibliographic data for series maintained by digitalhealth ().