Benchmarking the most popular XAI used for explaining clinical predictive models: Untrustworthy but could be useful.

IF 2.2 3区 医学 Q2 HEALTH CARE SCIENCES & SERVICES Health Informatics Journal Pub Date : 2024-10-01 DOI:10.1177/14604582241304730
Aida Brankovic, David Cook, Jessica Rahman, Sankalp Khanna, Wenjie Huang
{"title":"Benchmarking the most popular XAI used for explaining clinical predictive models: Untrustworthy but could be useful.","authors":"Aida Brankovic, David Cook, Jessica Rahman, Sankalp Khanna, Wenjie Huang","doi":"10.1177/14604582241304730","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>This study aimed to assess the practicality and trustworthiness of explainable artificial intelligence (XAI) methods used for explaining clinical predictive models.</p><p><strong>Methods: </strong>Two popular XAIs used for explaining clinical predictive models were evaluated based on their ability to generate domain-appropriate representations, impact clinical workflow, and consistency. Explanations were benchmarked against true clinical deterioration triggers recorded in the data system and agreement was quantified. The evaluation was conducted using two Electronic Medical Records datasets from major hospitals in Australia. Results were examined and commented on by a senior clinician.</p><p><strong>Results: </strong>Findings demonstrate a violation of consistency criteria and moderate concordance (0.47-0.8) with true triggers, undermining reliability and actionability, criteria for clinicians' trust in XAI.</p><p><strong>Conclusion: </strong>Explanations are not trustworthy to guide clinical interventions, though they may offer useful insights and help model troubleshooting. Clinician-informed XAI development and presentation, clear disclaimers on limitations, and critical clinical judgment can promote informed decisions and prevent over-reliance.</p>","PeriodicalId":55069,"journal":{"name":"Health Informatics Journal","volume":"30 4","pages":"14604582241304730"},"PeriodicalIF":2.2000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Health Informatics Journal","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1177/14604582241304730","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Objective: This study aimed to assess the practicality and trustworthiness of explainable artificial intelligence (XAI) methods used for explaining clinical predictive models.

Methods: Two popular XAIs used for explaining clinical predictive models were evaluated based on their ability to generate domain-appropriate representations, impact clinical workflow, and consistency. Explanations were benchmarked against true clinical deterioration triggers recorded in the data system and agreement was quantified. The evaluation was conducted using two Electronic Medical Records datasets from major hospitals in Australia. Results were examined and commented on by a senior clinician.

Results: Findings demonstrate a violation of consistency criteria and moderate concordance (0.47-0.8) with true triggers, undermining reliability and actionability, criteria for clinicians' trust in XAI.

Conclusion: Explanations are not trustworthy to guide clinical interventions, though they may offer useful insights and help model troubleshooting. Clinician-informed XAI development and presentation, clear disclaimers on limitations, and critical clinical judgment can promote informed decisions and prevent over-reliance.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
对用于解释临床预测模型的最流行的XAI进行基准测试:不可信但可能有用。
目的:本研究旨在评估可解释人工智能(XAI)方法用于解释临床预测模型的实用性和可信度。方法:对两种常用的用于解释临床预测模型的xai进行评估,基于它们生成适合领域的表示、影响临床工作流程和一致性的能力。根据数据系统中记录的真实临床恶化触发因素对解释进行基准测试,并对一致性进行量化。评估使用了澳大利亚各大医院的两个电子医疗记录数据集。结果由一位资深临床医生检查和评论。结果:研究结果表明,与真实触发器的一致性标准和中度一致性(0.47-0.8)不符合,破坏了临床医生对XAI的信任标准的可靠性和可操作性。结论:尽管解释可能提供有用的见解并有助于建立故障诊断模型,但它不能可靠地指导临床干预。临床知情的XAI开发和呈现、明确的局限性免责声明和关键的临床判断可以促进知情决策,防止过度依赖。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Health Informatics Journal
Health Informatics Journal HEALTH CARE SCIENCES & SERVICES-MEDICAL INFORMATICS
CiteScore
7.80
自引率
6.70%
发文量
80
审稿时长
6 months
期刊介绍: Health Informatics Journal is an international peer-reviewed journal. All papers submitted to Health Informatics Journal are subject to peer review by members of a carefully appointed editorial board. The journal operates a conventional single-blind reviewing policy in which the reviewer’s name is always concealed from the submitting author.
期刊最新文献
Researching public health datasets in the era of deep learning: a systematic literature review. A blueprint for large language model-augmented telehealth for HIV mitigation in Indonesia: A scoping review of a novel therapeutic modality. Advancing African American and hispanic health literacy with a bilingual, personalized, prevention smartphone application. Evaluating the quality of Spanish-language information for patients with type 2 diabetes on YouTube and Facebook. Pathways to usage intention of mobile health apps among hypertensive patients: A fuzzy-set qualitative comparative analysis.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1