Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography.

IF 9.7 1区医学 Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING Radiologia Medica Pub Date : 2025-02-12 DOI:10.1007/s11547-025-01964-6

Su Hwan Kim, Severin Schramm, Evamaria Olga Riedel, Lena Schmitzer, Enrike Rosenkranz, Olivia Kertels, Jannis Bodden, Karolin Paprottka, Dominik Sepp, Martin Renz, Jan Kirschke, Thomas Baum, Christian Maegerlein, Tobias Boeckh-Behrens, Claus Zimmer, Benedikt Wiestler, Dennis M Hedderich

{"title":"Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography.","authors":"Su Hwan Kim, Severin Schramm, Evamaria Olga Riedel, Lena Schmitzer, Enrike Rosenkranz, Olivia Kertels, Jannis Bodden, Karolin Paprottka, Dominik Sepp, Martin Renz, Jan Kirschke, Thomas Baum, Christian Maegerlein, Tobias Boeckh-Behrens, Claus Zimmer, Benedikt Wiestler, Dennis M Hedderich","doi":"10.1007/s11547-025-01964-6","DOIUrl":null,"url":null,"abstract":"Purpose: To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.Material and methods: Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.Results: False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.Conclusion: Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.","PeriodicalId":20817,"journal":{"name":"Radiologia Medica","volume":" ","pages":""},"PeriodicalIF":9.7000,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Radiologia Medica","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s11547-025-01964-6","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.

Material and methods: Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.

Results: False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.

Conclusion: Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

求助全文

约1分钟内获得全文去求助

来源期刊

Radiologia Medica 医学-核医学

CiteScore

14.10

自引率

7.90%

发文量

133

审稿时长

4-8 weeks

期刊介绍： Felice Perussia founded La radiologia medica in 1914. It is a peer-reviewed journal and serves as the official journal of the Italian Society of Medical and Interventional Radiology (SIRM). The primary purpose of the journal is to disseminate information related to Radiology, especially advancements in diagnostic imaging and related disciplines. La radiologia medica welcomes original research on both fundamental and clinical aspects of modern radiology, with a particular focus on diagnostic and interventional imaging techniques. It also covers topics such as radiotherapy, nuclear medicine, radiobiology, health physics, and artificial intelligence in the context of clinical implications. The journal includes various types of contributions such as original articles, review articles, editorials, short reports, and letters to the editor. With an esteemed Editorial Board and a selection of insightful reports, the journal is an indispensable resource for radiologists and professionals in related fields. Ultimately, La radiologia medica aims to serve as a platform for international collaboration and knowledge sharing within the radiological community.