An interpretable hybrid machine learning approach for predicting three-month unfavorable outcomes in patients with acute ischemic stroke

Background

Acute ischemic stroke (AIS) is a clinical disorder caused by nontraumatic cerebrovascular disease with a high incidence, mortality, and disability rate. Most stroke survivors are left with speech and physical impairments, and emotional problems. Despite technological advances and improved treatment options, death and disability after stroke remain a major problem. Our research aims to develop interpretable hybrid machine learning (ML) models to accurately predict three-month unfavorable outcomes in patients with AIS.

Methods

Within the framework of this analysis, the model was trained using data from 731 cases in the dataset and subsequently validated using data from both internal and external validation datasets. A total of 25 models (including ML and deep learning models) were initially employed, along with 14 evaluation metrics, and the results were subjected to cluster analysis to objectively validate the model’s effectiveness and assess the similarity of evaluation metrics. For the final model evaluation, 10 metrics selected after metric screening and calibration analysis were utilized to evaluate model performance, while clinical decision analysis, cost curve analysis, and model fairness analysis were applied to assess the clinical applicability of the model. Nested cross-validation and optimal hyperparameter search were employed to determine the best hyperparameter for the ML models. The SHAP diagram is utilized to provide further visual explanations regarding the importance of features and their interaction effects, ultimately leading to the establishment of a practical AIS three-month prognostic prediction platform.

Results

The frequencies of unfavorable outcomes in the internal dataset and external validation dataset were 389 / 1045 (37.2 %) and 161 / 411 (39.2 %), respectively. Through cluster analysis of the results of 14 evaluation metrics across 25 models and a comparison of clinical applicability, 12 ML models were ultimately selected for further analysis. The findings revealed that XGBoost and CatBoost performed the best. Further ensemble modeling of these two models and adjustment of decision thresholds using cost curves resulted in the final model performing as follows on the internal validation set: PRAUC of 0.856 (0.801, 0.902), ROCAUC of 0.856 (0.801, 0.901), specificity of 0.879 (0.797, 0.953), balanced accuracy of 0.840 (0.763, 0.912) and MCC of 0.678 (0.591, 0.760). Similarly, the model exhibited excellent performance on the external validation set, with a PRAUC of 0.823 (0.775, 0.872), ROCAUC of 0.842 (0.801, 0.890), specificity of 0.888 (0.822, 0.920), balanced accuracy of 0.814 (0.751, 0.869) and MCC of 0.639 (0.546, 0.721). In terms of the important features of AIS three-month outcomes, albumin ranked highest, followed by FBG, BMI, Scr, WBC, and age, while gender exhibited significant interactions with other indicators. Ultimately, based on the final ensemble model and optimal decision thresholds, a tailored short-term prognostic prediction platform for AIS patients was developed.

Conclusions

We constructed an interpretable hybrid ML model that maintained good performance on both internal and external validation datasets using the most readily accessible 30 clinical data variables, indicating its ability to accurately predict the three-month unfavorable outcomes for AIS patients. Meanwhile, our superior predictive model provides practicality for routine and more frequent initial risk assessments, making it easier to integrate into network or mobile-based telemedicine solutions.