A Machine Learning–Enabled Partially Observable Markov Decision Process Framework for Early Sepsis Prediction

Zeyu Liu (), Anahita Khojandi (), Xueping Li (), Akram Mohammed (), Robert L Davis () and Rishikesan Kamaleswaran ()
Additional contact information
Zeyu Liu: Department of Industrial and Systems Engineering, University of Tennessee, Knoxville, Tennessee 37996
Anahita Khojandi: Department of Industrial and Systems Engineering, University of Tennessee, Knoxville, Tennessee 37996
Xueping Li: Department of Industrial and Systems Engineering, University of Tennessee, Knoxville, Tennessee 37996
Akram Mohammed: Center for Biomedical Informatics-Pediatrics, University of Tennessee Health Science Center, Memphis, Tennessee 38163
Robert L Davis: Center for Biomedical Informatics-Pediatrics, University of Tennessee Health Science Center, Memphis, Tennessee 38163
Rishikesan Kamaleswaran: Departments of Biomedical Informatics, Pediatrics, and Emergency Medicine, Emory University School of Medicine, Atlanta, Georgia 30322

INFORMS Journal on Computing, 2022, vol. 34, issue 4, 2039-2057

Abstract: Sepsis is a life-threatening condition, caused by the body’s extreme response to an infection. In the United States, 1.7 million cases of sepsis occur annually, resulting in 265,000 deaths. Delayed diagnosis and treatment are associated with higher mortality rates. An exponential rise in the availability of medical data has allowed for the development of sophisticated machine learning algorithms to predict sepsis earlier than the onset. However, these models often underperform, as the training data are retrospective and do not fully capture the uncertain future. In this study, we develop a novel framework, which we refer to as MLePOMDP , to leverage and combine the underlying, high-level knowledge about sepsis progression and machine learning (ML) for classification. Specifically, we use a hidden Markov model to describe sepsis development at a high level, where the ML model makes the higher-order “observations” from temporal data. Consequently, a partially observable Markov decision process (POMDP) model is developed to make classification decisions. We analytically establish that the optimal policy is of threshold-type, which we exploit to efficiently optimize MLePOMDP. MLePOMDP is calibrated and tested using high-frequency physiological data collected from bedside monitors. Different from past POMDP-based frameworks, MLePOMDP is developed for a prediction task using a very small state definition, produces highly interpretable results, and accounts for a novel and clinically meaningful action space. Our results show that MLePOMDP outperforms machine learning–based benchmarks by up to 8% in precision. Importantly, MLePOMDP is able to reduce false alarms by up to 28%. An additional experiment is conducted to show the generalizability of MLePOMDP to different patient cohorts. Summary of Contribution: This study develops a novel real-time decision support framework for early sepsis prediction by integrating well-known machine learning models (random forest and neural networks) with a well-established sequential decision-making model, namely, a partially observable Markov decision process (POMDP). The structural properties of the optimal policy are further explored and a threshold-type structure is established, which is then leveraged to develop a customized algorithm to solve the problem more efficiently. The resulting framework demonstrates the benefit of applying POMDPs to augment machine learning outputs. Specifically, the framework results in the reduction of false alarms in sepsis predictions where decisions are made in real time, hence improving the overall prediction precision.

Keywords: partially observable Markov decision processes; medical decision making; sepsis; real-time predictive analytics; hierarchical modeling (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/ijoc.2022.1176 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orijoc:v:34:y:2022:i:4:p:2039-2057

Access Statistics for this article

More articles in INFORMS Journal on Computing from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().