Optimizing the First Response to Sepsis: An Electronic Health Record-Based Markov Decision Process Model

Sepsis is considered a medical emergency where delays in initial treatment are associated with increased morbidity and mortality, yet there is no gold standard for identifying sepsis onset and thus treatment timing. We leverage electronic health record (EHR) data with clinical expertise to develop a continuous-time Markov decision process (MDP) optimal stopping model that identifies the optimal first intervention action (anti-infective, fluid, or wait). To study the impact of initial treatment of patients at risk for developing sepsis, we define the delayed treatment population who received delayed treatment upon admission or during hospitalization and serves as an approximation of the natural history of sepsis. We apply the optimal first treatment policy to sample patient visits from the nondelayed treatment population. This analysis indicates the average risk of death could be reduced by approximately 2.2%, the average time until treatment could be reduced by 106 minutes, and the average severity of the treatment state could be reduced by 15.5% compared with the treatment they received in the hospital. We study the properties of the optimal policy to define an easily interpretable initial treatment heuristic that considers a patient’s organ dysfunction, location, and septic shock status. This generalizable framework can inform personalized treatment of patients at risk for sepsis.