Implementation of an AI Algorithm in Clinical Practice to Reduce Missed Incidental Pulmonary Embolisms on Chest CT and Its Impact on Short-Term Survival.

Abstract

Objectives: A substantial number of incidental pulmonary embolisms (iPEs) in computed tomography scans are missed by radiologists in their daily routine. This study analyzes the radiological reports of iPE cases before and after implementation of an artificial intelligence (AI) algorithm for iPE detection. Furthermore, we investigate the anatomic distribution patterns within missed iPE cases and mortality within a 90-day follow-up in patients before and after AI use.

Materials and methods: This institutional review board-approved observational single-center study included 5298 chest computed tomography scans performed for reasons other than suspected pulmonary embolism (PE). We compared 2 cohorts: cohort 1, consisting of 1964 patients whose original radiology reports were generated before the implementation of an AI algorithm, and cohort 2, consisting of 3334 patients whose scans were analyzed after the implementation of an Food and Drug Administration-approved and CE-certified AI algorithm for iPE detection (Aidoc Medical, Tel Aviv, Israel). For both cohorts, any discrepancies between the original radiology reports and the AI results were reviewed by 2 thoracic imaging subspecialized radiologists. In the original radiology report and in case of discrepancies with the AI algorithm, the expert review served as reference standard. Sensitivity, specificity, prevalence, negative predictive value (NPV), and positive predictive value (PPV) were calculated. The rates of missed iPEs in both cohorts were compared statistically using STATA (Version 17.1). Kaplan-Meier curves and Cox proportional hazards models were used for survival analysis.

Results: In cohort 1 (mean age 70.6 years, 48% female [n = 944], 52% male [n = 1020]), the prevalence of confirmed iPE was 2.2% (n = 42), and the AI detected 61 suspicious iPEs, resulting in a sensitivity of 95%, a specificity of 99%, a PPV of 69%, and an NPV of 99%. Radiologists missed 50% of iPE cases in cohort 1. In cohort 2 (mean age 69 years, 47% female [n = 1567], 53% male [n = 1767]), the prevalence of confirmed iPEs was 1.7% (56/3334), with AI detecting 59 suspicious cases (sensitivity 90%, specificity 99%, PPV 95%, NPV 99%). The rate of missed iPEs by radiologists dropped to 7.1% after AI implementation, showing a significant improvement (P < 0.001). Most overlooked iPEs (61%) were in the right lower lobe. The survival analysis showed no significantly decreased 90-day mortality rate, with a hazards ratio of 0.95 (95% confidence interval, 0.45-1.96; P = 0.88).

Conclusions: The implementation of an AI algorithm significantly reduced the rate of missed iPEs from 50% to 7.1%, thereby enhancing diagnostic accuracy. Despite this improvement, the 90-day mortality rate remained unchanged. These findings highlight the AI tool's potential to assist radiologists in accurately identifying iPEs, although its implementation does not significantly affect short-term survival. Notably, most missed iPEs were located in the right lower lobe, suggesting that radiologists should pay particular attention to this area during evaluations.