Machine Learning-Based Pediatric Early Warning Score: Patient Outcomes in a Pre- Versus Post-Implementation Study, 2019-2023.

Abstract

Objectives: To describe the deployment of pediatric Calculated Assessment of Risk and Triage (pCART), a machine learning (ML) model to predict the risk of the direct ward to the ICU transfer within 12 hours, and the associated improved outcomes among hospitalized children.

Design: Pre- vs. post-implementation study.

Setting: An urban, tertiary-care, academic hospital.

Patients: Pediatric (age < 18 yr) admissions from May 1, 2019, to April 30, 2023.

Interventions: None.

Measurements and main results: Patients were divided into baseline, pre-pCART implementation (May 1, 2019, to April 30 2021), and post-pCART implementation (May 1, 2021, to April 30, 2023) cohorts. First-ward admissions with a high-risk score (pCART score ≥ 92) were considered as the main cohort. The primary outcome was the occurrence of critical events, defined as invasive mechanical ventilation, vasoactive drug administration, or death within 12 hours of the first high-risk pCART score. There were 2763 and 3943 patients in the baseline and implementation cohorts, respectively. pCART implementation was associated with a decrease in the percentage of the primary outcome from baseline 1.4% to 0.4% (p < 0.001), which converted to more than two-thirds lower adjusted odds of the primary outcome (odds ratio, 0.22 [95% CI, 0.11-0.40]; p < 0.001). pCART implementation was also associated with a decreased prevalence of critical events at 24 and 48 hours after a first high-risk score. We failed to identify any association between cohort period and overall hospital and ICU length-of-stay, number of ICU transfers, and time to ICU transfer. However, there was a difference in hospital length-of-stay among a subpopulation of patients transferred to the ICU (median 6 vs. 7 d; p < 0.001). Analysis of compliance metrics indicates sustained compliance achievements over time.

Conclusions: The deployment of pCART, a ML-based pediatric risk stratification tool, for clinical decision support for pediatric ward patients, was associated with lower odds of critical events among high-risk patients.