External validation of the HACOR score and ROX index for predicting treatment failure in patients with coronavirus disease 2019 pneumonia managed on high-flow nasal cannula therapy: a multicenter retrospective observational study in Japan.

Abstract

Background: The HACOR score for predicting treatment failure includes vital signs and acid-base balance factors, whereas the ROX index only considers the respiratory rate, oxygen saturation, and fraction of inspired oxygen (FiO₂). We aimed to externally validate the HACOR score and ROX index for predicting treatment failure in patients with coronavirus disease 2019 (COVID-19) on high-flow nasal cannula (HFNC) therapy in Japan.

Methods: This retrospective, observational, multicenter study included patients, aged ≥ 18 years, diagnosed with COVID-19 and treated with HFNC therapy between January 16, 2020, and March 31, 2022. The HACOR score and ROX index were calculated at 2, 6, 12, 24, and 48 h after stating HFNC therapy. The primary outcome was treatment failure (requirement for intubation or occurrence of death within 7 days). We calculated the area under the receiver operating characteristic curve (AUROC) and assessed the diagnostic performance of these indicators. The 2-h time-point prediction was considered the primary analysis and that of other time-points as the secondary analysis. We also assessed 2-h time-point sensitivity and specificity using previously reported cutoff values (HACOR score > 5, ROX index < 2.85).

Results: We analyzed 300 patients from 9 institutions (median age, 60 years; median SpO₂/FiO₂ ratio at the start of HFNC therapy, 121). Within 7 days of HFNC therapy, treatment failure occurred in 127 (42%) patients. The HACOR score and ROX index at the 2-h time-point exhibited AUROC discrimination values of 0.63 and 0.57 (P = 0.24), respectively. These values varied with temporal changes-0.58 and 0.62 at 6 h, 0.70 and 0.68 at 12 h, 0.68 and 0.69 at 24 h, and 0.75 and 0.75 at 48 h, respectively. The 2-h time-point sensitivity and specificity were 18% and 91% for the HACOR score, respectively, and 3% and 100% for the ROX index, respectively. Visual calibration assessment revealed well calibrated HACOR score, but not ROX index.

Conclusions: In COVID-19 patients receiving HFNC therapy in Japan, the predictive performance of the HACOR score and ROX index at the 2-h time-point may be inadequate. Furthermore, clinicians should be mindful of time-point scores owing to the variation of the models' predictive performance with the time-point. Trial registration UMIN (registration number: UMIN000050024, January 13, 2023).