Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography.

IF 4.8 1区医学 Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING Radiologia Medica Pub Date : 2025-04-01 Epub Date: 2025-02-12 DOI:10.1007/s11547-025-01964-6

Su Hwan Kim, Severin Schramm, Evamaria Olga Riedel, Lena Schmitzer, Enrike Rosenkranz, Olivia Kertels, Jannis Bodden, Karolin Paprottka, Dominik Sepp, Martin Renz, Jan Kirschke, Thomas Baum, Christian Maegerlein, Tobias Boeckh-Behrens, Claus Zimmer, Benedikt Wiestler, Dennis M Hedderich

{"title":"Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography.","authors":"Su Hwan Kim, Severin Schramm, Evamaria Olga Riedel, Lena Schmitzer, Enrike Rosenkranz, Olivia Kertels, Jannis Bodden, Karolin Paprottka, Dominik Sepp, Martin Renz, Jan Kirschke, Thomas Baum, Christian Maegerlein, Tobias Boeckh-Behrens, Claus Zimmer, Benedikt Wiestler, Dennis M Hedderich","doi":"10.1007/s11547-025-01964-6","DOIUrl":null,"url":null,"abstract":"Purpose: To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.Material and methods: Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.Results: False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.Conclusion: Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.","PeriodicalId":20817,"journal":{"name":"Radiologia Medica","volume":" ","pages":"555-566"},"PeriodicalIF":4.8000,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12008054/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Radiologia Medica","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s11547-025-01964-6","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/2/12 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.

Material and methods: Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.

Results: False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.

Conclusion: Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.

Abstract Image

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

在飞行时间磁共振血管造影中人工智能辅助检测脑动脉瘤的自动化偏差。

目的：确定自动化偏差（人类对过度信任自动化决策系统的倾向）如何影响放射科医生在解释飞行时间磁共振血管造影（TOF-MRA）研究中人工智能检测到的脑动脉瘤结果。材料和方法：9名具有不同经验水平的放射科医生评估了20例TOF-MRA检查是否存在脑动脉瘤。在人工智能软件©mdbrain的帮助下和不帮助下对每个病例进行评估，其间至少有四周的洗脱期。一半的病例包括至少一个假阳性的人工智能发现。动脉瘤评分、随访建议和阅读时间采用Wilcoxon sign -rank检验进行评估。结果：假阳性的AI结果明显提高了动脉瘤的怀疑程度（p = 0.01）。缺乏经验的读者进一步建议，当出现人工智能假阳性结果时，应进行更严格的随访检查（p = 0.005）。人工智能辅助下的阅读时间明显缩短（164.1秒vs 228.2秒）；结论：我们的研究结果表明，在TOF-MRA研究中，当遇到假阳性的AI结果时，放射学读者在检测脑动脉瘤时容易受到自动化偏差的影响。虽然用于脑动脉瘤检测的人工智能系统可以提供好处，但需要减轻人类与人工智能交互方面的挑战，以确保安全有效地采用。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Radiologia Medica 医学-核医学

CiteScore

14.10

自引率

7.90%

发文量

133

审稿时长

4-8 weeks

期刊介绍： Felice Perussia founded La radiologia medica in 1914. It is a peer-reviewed journal and serves as the official journal of the Italian Society of Medical and Interventional Radiology (SIRM). The primary purpose of the journal is to disseminate information related to Radiology, especially advancements in diagnostic imaging and related disciplines. La radiologia medica welcomes original research on both fundamental and clinical aspects of modern radiology, with a particular focus on diagnostic and interventional imaging techniques. It also covers topics such as radiotherapy, nuclear medicine, radiobiology, health physics, and artificial intelligence in the context of clinical implications. The journal includes various types of contributions such as original articles, review articles, editorials, short reports, and letters to the editor. With an esteemed Editorial Board and a selection of insightful reports, the journal is an indispensable resource for radiologists and professionals in related fields. Ultimately, La radiologia medica aims to serve as a platform for international collaboration and knowledge sharing within the radiological community.