External Validation of Eight Ruptured Abdominal Aortic Aneurysm Mortality Prediction Models Demonstrates Limited Predictive Accuracy

Objective

Over a dozen ruptured abdominal aortic aneurysm (rAAA) mortality risk prediction models currently exist; however, lack of external validation limits their applicability. This study aimed to evaluate the accuracy of eight common rAAA mortality risk prediction models in a large, contemporary, external validation cohort.

Methods

A retrospective review of rAAA repairs at a multicentre integrated regional healthcare system with large central quaternary referral facility (2010 – 2020) was performed. Eight models were used to predict 30 day post-operative death, including the Updated Glasgow Aneurysm Score (GAS), Vascular Study Group of New England rAAA Risk Score, Harborview Pre-operative rAAA Risk Score, Modified Harborview Risk Score, Vancouver Scoring System (VSS), Artificial Neural Network Score, Dutch Aneurysm Score, and Edinburgh Ruptured Aneurysm Score. The models were assessed for discrimination, calibration, and clinical utility using receiver operating characteristic curves (area under the curve [AUC]), Hosmer–Lemeshow χ² test, Brier scores, and decision curve analysis. The proportion of unexpected survivors (survival despite > 80% predicted 30 day death) to expected deaths was compared across calculators, and both groups were compared using the model demonstrating the highest unexpected survival frequency.

Results

Three hundred and fifteen rAAA repairs were included (mean age 73.6 ± 10.0 years; 72.1% male; 49.8% open repair) with a 30 day mortality rate of 32.1%. Three models had fair discrimination (AUC ≥ 0.70), with GAS having the highest AUC (0.74, 95% confidence interval 0.68 – 0.79). All models demonstrated poor to adequate calibration. Using VSS, unexpected survivors (n = 25) had less pre-operative shock (72% vs. 96%; p = .050) and statistically significantly less coagulopathy (median international normalised ratio 1.2 [interquartile range 1.1, 1.5] vs. 1.8 [1.3, 2.2]; p = .015) compared with expected deaths (n = 23).

Conclusion

Current rAAA risk prediction models demonstrated only fair discrimination and poor to adequate calibration. These findings suggest that existing risk prediction models have not sufficiently captured important physiological characteristics associated with rAAA death and should be applied cautiously to clinical practice.