Assessing the Performance of Artificial Intelligence Assistance for Prostate MRI: A Two-Center Study Involving Radiologists With Different Experience Levels.

Abstract

Background: Artificial intelligence (AI) assistance may enhance radiologists' performance in detecting clinically significant prostate cancer (csPCa) on MRI. Further validation is needed for radiologists with different experiences.

Purpose: To assess the performance of experienced and less-experienced radiologists in detecting csPCa, with and without AI assistance.

Study type: Retrospective.

Population: Nine hundred patients who underwent prostate MRI and biopsy (median age 67 years; 356 with csPCa and 544 with non-csPCa).

Field strength/sequence: 3-T and 1.5-T, diffusion-weighted imaging using a single-shot gradient echo-planar sequence, turbo spin echo T2-weighted image.

Assessment: CsPCa regions based on biopsy results served as the reference standard. Ten less-experienced (<500 prostate MRIs) and six experienced (>1000 prostate MRIs) radiologists reviewed each case twice using Prostate Imaging Reporting and Data System v2.1, with and without AI, separated by 4-week intervals. Cases were equally distributed among less-experienced radiologists, and 90 cases were randomly assigned to each experienced radiologist. Reading time and diagnostic confidence were assessed.

Statistical tests: Area under the curve (AUC), sensitivity, specificity, reading time, and diagnostic confidence were compared using the DeLong test, Chi-squared test, Fisher exact test, or Wilcoxon rank-sum test between the two sessions. A P-value <0.05 was considered significant. Adjusting threshold using Bonferroni correction was performed for multiple comparisons.

Results: For less-experienced radiologists, AI assistance significantly improved lesion-level sensitivity (0.78 vs. 0.88), sextant-level AUC (0.84 vs. 0.93), and patient-level AUC (0.84 vs. 0.89). For experienced radiologists, AI assistance only improved sextant-level AUC (0.82 vs. 0.91). AI assistance significantly reduced median reading time (250 s [interquartile range, IQR: 157, 402] vs. 130 s [IQR: 88, 209]) and increased diagnostic confidence (5 [IQR: 4, 5] vs. 5 [IQR: 4, 5]) irrespective of experience and enhanced consistency among experienced radiologists (Fleiss κ: 0.53 vs. 0.61).

Data conclusion: AI-assisted reading improves the performance of detecting csPCa on MRI, particularly for less-experienced radiologists.

Evidence level: 3 TECHNICAL EFFICACY: Stage 2.