Effect of Artificial Intelligence as a Second Reader on the Lung Nodule Detection and Localization Accuracy of Radiologists and Non-radiology Physicians in Chest Radiographs: A Multicenter Reader Study.

Abstract

Rationale and objectives: Missed nodules in chest radiographs (CXRs) are common occurrences. We assessed the effect of artificial intelligence (AI) as a second reader on the accuracy of radiologists and non-radiology physicians in lung nodule detection and localization in CXRs.

Materials and methods: This retrospective study using the multi-reader multi-case design included 300 CXRs acquired from 40 hospitals across the US. All CXRs had a paired follow-up image (chest CT or CXR) to augment the ground truth establishment for the presence and location of nodules on CXRs by five independent thoracic radiologists. 15 readers (nine radiologists and six non-radiology physicians) read each CXR twice in a second-reader paradigm, once without AI and then immediately with AI assistance. The primary analysis assessed the difference in area-under-the-alternative-free-response-receiver-operating-characteristic-curve (AFROC) of readers with and without AI. Case-level area-under-the-receiver-operating-characteristic-curve (AUROC), sensitivity, and specificity were assessed in secondary analyses.

Results: A total of 300 CXRs (147 with nodules, 153 without nodules) from 300 patients (mean age, 64 years ± 15 [standard deviation]; 174 women) were included. The mean AFROC of readers was 0.73 without AI and 0.81 with AI (95% CI of difference, 0.05-0.10). Case-level AUROC was 0.77 without AI and 0.84 with AI (95% CI of difference, 0.04-0.09). Case-level sensitivity was 72.8% and 83.5% (95% CI of difference, 6.8-14.6) and specificity was 71.1% and 72.0% (95% CI of difference, -0.8-2.6) without and with AI, respectively.

Conclusion: Using AI, readers detected and localized more nodules without any significant difference in false positive interpretations.