Prediction of mortality in heart failure by machine learning. Comparison with statistical modeling

Background

Assessing the relative performance of machine learning (ML) methods and conventional statistical methods in predicting prognosis in heart failure (HF) still remains a challenging research field.

Methods

The primary outcome was three-year mortality. The following 5 machine learning approaches were used for modeling: Random Forest (RF), Gradient Boosting, Extreme Gradient Boosting (XGBoost), Support Vector Machine, and Multilayer perceptron. We compared the performance of the best performing ML models to the MAGGIC (Meta-analysis Global Group in Chronic Heart Failure) score and a novel logistic regression model (LRM) developed using the same set of variables used to develop the machine learning models. The performance was determined based on discrimination, calibration, and net benefit.

Results

The XGBoost and the RF were the best performing ML models. The XGBoost provided the highest discrimination (C-statistic: 0.793) and the lowest Brier score (0.178); the RF model had a C-statistic of 0.779 and provided the highest area under the precision‐recall curve (0.636). Both models were well calibrated. Both the XGboost and RF models outperformed MAGGIC score. The LRM had a C-statistic of 0.811 and a Brier score of 0.160 and was well calibrated. The XGBoost, RF, and LRM gave a higher net benefit than MAGGIC score; the XGBoost, RF, and logistic regression model gave similar net benefit.

Conclusions

RF and XGBoost models outperformed MAGGIC in predicting mortality. However, they did not offer any improvement over a logistic regression model built using the same set of covariates considered in the ML modeling.