Impact of Surge Strain and Pandemic Progression on Prognostication by an Established COVID-19–Specific Severity Score

Abstract

IMPORTANCE: Many U.S. State crisis standards of care (CSC) guidelines incorporated Sequential Organ Failure Assessment (SOFA), a sepsis-related severity score, in pandemic triage algorithms. However, SOFA performed poorly in COVID-19. Although disease-specific scores may perform better, their prognostic utility over time and in overcrowded care settings remains unclear. OBJECTIVES: We evaluated prognostication by the modified 4C (m4C) score, a COVID-19–specific prognosticator that demonstrated good predictive capacity early in the pandemic, as a potential tool to standardize triage across time and hospital-surge environments. DESIGN: Retrospective observational cohort study. SETTING: Two hundred eighty-one U.S. hospitals in an administrative healthcare dataset. PARTICIPANTS: A total of 298,379 hospitalized adults with COVID-19 were identified from March 1, 2020, to January 31, 2022. m4C scores were calculated from admission diagnosis codes, vital signs, and laboratory values. MAIN OUTCOMES AND MEASURES: Hospital-surge index, a severity-weighted measure of COVID-19 caseload, was calculated for each hospital-month. Discrimination of in-hospital mortality by m4C and surge index-adjusted models was measured by area under the receiver operating characteristic curves (AUC). Calibration was assessed by training models on early pandemic waves and measuring fit (deviation from bisector) in subsequent waves. RESULTS: From March 2020 to January 2022, 298,379 adults with COVID-19 were admitted across 281 U.S. hospitals. m4C adequately discriminated mortality in wave 1 (AUC 0.779 [95% CI, 0.769–0.789]); discrimination was lower in subsequent waves (wave 2: 0.772 [95% CI, 0.765–0.779]; wave 3: 0.746 [95% CI, 0.743–0.750]; delta: 0.707 [95% CI, 0.702–0.712]; omicron: 0.729 [95% CI, 0.721–0.738]). m4C demonstrated reduced calibration in contemporaneous waves that persisted despite periodic recalibration. Performance characteristics were similar with and without adjustment for surge. CONCLUSIONS AND RELEVANCE: Mortality prediction by the m4C score remained robust to surge strain, making it attractive for when triage is most needed. However, score performance has deteriorated in recent waves. CSC guidelines relying on defined prognosticators, especially for dynamic disease processes like COVID-19, warrant frequent reappraisal to ensure appropriate resource allocation.