Improving breast ultrasonography education: the impact of AI-based decision support on the performance of non-specialist medical professionals.

IF 2.5 3区医学 Q2 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING Ultrasonography Pub Date : 2025-03-01 Epub Date: 2024-12-12 DOI:10.14366/usg.24171

Sangwon Lee, Hye Sun Lee, Eunju Lee, Won Hwa Kim, Jaeil Kim, Jung Hyun Yoon

{"title":"Improving breast ultrasonography education: the impact of AI-based decision support on the performance of non-specialist medical professionals.","authors":"Sangwon Lee, Hye Sun Lee, Eunju Lee, Won Hwa Kim, Jaeil Kim, Jung Hyun Yoon","doi":"10.14366/usg.24171","DOIUrl":null,"url":null,"abstract":"Purpose: This study evaluated the educational impact of an artificial intelligence (AI)-based decision support system for breast ultrasonography (US) on medical professionals not specialized in breast imaging.Methods: In this multi-case, multi-reader study, educational materials, including American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) descriptors, were provided alongside corresponding AI results during training. The AI system presented results in the form of AIheatmaps, AI scores, and AI-provided BI-RADS assessment categories. Forty-two readers evaluated the test set in three sessions: the first session (S1) occurred before the educational intervention, the second session (S2) followed education without AI assistance, and the third session (S3) took place after education with AI assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and overall performance, were compared between the sessions.Results: The mean sensitivity increased from 66.5% (95% confidence interval [CI], 59.2% to 73.7%) to 88.7% (95% CI, 84.1% to 93.3%), with a statistically significant difference (P<0.001), and the AUC non-significantly increased from 0.664 (95% CI, 0.606 to 0.723) to 0.684 (95% CI, 0.620 to 0.748) (P=0.300). Both measures were higher in S2 than in S1. The AI-achieved AUC was comparable to that of the expert reader (0.747 [95% CI, 0.640 to 0.855] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.217). Additionally, with AI assistance, the mean AUC for inexperienced readers was not significantly different from that of the expert reader (0.745 [95% CI, 0.660 to 0.830] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.120).Conclusion: The mean AUC and sensitivity improved after incorporating AI into breast US education and interpretation. AI systems with high-level performance for breast US can potentially be used as educational tools in the interpretation of breast US images.","PeriodicalId":54227,"journal":{"name":"Ultrasonography","volume":" ","pages":"124-133"},"PeriodicalIF":2.5000,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11938798/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ultrasonography","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.14366/usg.24171","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/12/12 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: This study evaluated the educational impact of an artificial intelligence (AI)-based decision support system for breast ultrasonography (US) on medical professionals not specialized in breast imaging.

Methods: In this multi-case, multi-reader study, educational materials, including American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) descriptors, were provided alongside corresponding AI results during training. The AI system presented results in the form of AIheatmaps, AI scores, and AI-provided BI-RADS assessment categories. Forty-two readers evaluated the test set in three sessions: the first session (S1) occurred before the educational intervention, the second session (S2) followed education without AI assistance, and the third session (S3) took place after education with AI assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and overall performance, were compared between the sessions.

Results: The mean sensitivity increased from 66.5% (95% confidence interval [CI], 59.2% to 73.7%) to 88.7% (95% CI, 84.1% to 93.3%), with a statistically significant difference (P<0.001), and the AUC non-significantly increased from 0.664 (95% CI, 0.606 to 0.723) to 0.684 (95% CI, 0.620 to 0.748) (P=0.300). Both measures were higher in S2 than in S1. The AI-achieved AUC was comparable to that of the expert reader (0.747 [95% CI, 0.640 to 0.855] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.217). Additionally, with AI assistance, the mean AUC for inexperienced readers was not significantly different from that of the expert reader (0.745 [95% CI, 0.660 to 0.830] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.120).

Conclusion: The mean AUC and sensitivity improved after incorporating AI into breast US education and interpretation. AI systems with high-level performance for breast US can potentially be used as educational tools in the interpretation of breast US images.

Abstract Image

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

改进乳腺超声教育：基于人工智能的决策支持对非专科医疗专业人员绩效的影响

目的：本研究评估了基于人工智能（AI）的乳腺超声检查决策支持系统（US）对非乳腺成像专业医疗人员的教育影响。方法：在这项多病例、多读者的研究中，在训练期间提供了包括美国放射学会乳腺成像报告和数据系统（BI-RADS）描述符在内的教育材料以及相应的人工智能结果。AI系统以AI热图、AI分数和AI提供的BI-RADS评估类别的形式呈现结果。42位读者分三个阶段对测试集进行评估：第一个阶段（S1）在教育干预之前进行，第二个阶段（S2）在没有人工智能帮助的情况下进行教育，第三个阶段（S3）在有人工智能帮助的教育之后进行。比较两组患者工作特征曲线下面积（AUC）、敏感性、特异性和总体表现。结果：平均敏感性从66.5%（95%可信区间[CI]， 59.2% ~ 73.7%）增加到88.7% (95% CI, 84.1% ~ 93.3%)，差异有统计学意义（P<0.001）， AUC从0.664 （95% CI, 0.606 ~ 0.723）增加到0.684 (95% CI, 0.620 ~ 0.748)，无统计学意义（P=0.300）。两项指标S2均高于S1。人工智能实现的AUC与专家读者相当（0.747 [95% CI， 0.640至0.855]对0.803 [95% CI， 0.706至0.900]，P=0.217）。此外，在人工智能的帮助下，没有经验的读者的平均AUC与专家读者的平均AUC没有显著差异（0.745 [95% CI， 0.660至0.830]vs. 0.803 [95% CI， 0.706至0.900]，P=0.120）。结论：人工智能应用于乳腺超声教育和解释后，平均AUC和灵敏度均有提高。具有高水平乳腺超声成像性能的人工智能系统可能被用作解读乳腺超声图像的教育工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Ultrasonography Medicine-Radiology, Nuclear Medicine and Imaging

CiteScore

5.10

自引率

6.50%

发文量

审稿时长

15 weeks

期刊介绍： Ultrasonography, the official English-language journal of the Korean Society of Ultrasound in Medicine (KSUM), is an international peer-reviewed academic journal dedicated to practice, research, technology, and education dealing with medical ultrasound. It is renamed from the Journal of Korean Society of Ultrasound in Medicine in January 2014, and published four times per year: January 1, April 1, July 1, and October 1. Original articles, technical notes, topical reviews, perspectives, pictorial essays, and timely editorial materials are published in Ultrasonography covering state-of-the-art content. Ultrasonography aims to provide updated information on new diagnostic concepts and technical developments, including experimental animal studies using new equipment in addition to well-designed reviews of contemporary issues in patient care. Along with running KSUM Open, the annual international congress of KSUM, Ultrasonography also serves as a medium for cooperation among physicians and specialists from around the world who are focusing on various ultrasound technology and disease problems and relevant basic science.