Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis.

IF 11.7 1区 医学 Q1 CELL BIOLOGY Cell Reports Medicine Pub Date : 2025-01-21 Epub Date: 2024-12-19 DOI:10.1016/j.xcrm.2024.101876
Yuanyuan Peng, Aidi Lin, Meng Wang, Tian Lin, Linna Liu, Jianhua Wu, Ke Zou, Tingkun Shi, Lixia Feng, Zhen Liang, Tao Li, Dan Liang, Shanshan Yu, Dawei Sun, Jing Luo, Ling Gao, Xinjian Chen, Ching-Yu Cheng, Huazhu Fu, Haoyu Chen
{"title":"Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis.","authors":"Yuanyuan Peng, Aidi Lin, Meng Wang, Tian Lin, Linna Liu, Jianhua Wu, Ke Zou, Tingkun Shi, Lixia Feng, Zhen Liang, Tao Li, Dan Liang, Shanshan Yu, Dawei Sun, Jing Luo, Ling Gao, Xinjian Chen, Ching-Yu Cheng, Huazhu Fu, Haoyu Chen","doi":"10.1016/j.xcrm.2024.101876","DOIUrl":null,"url":null,"abstract":"<p><p>Inability to express the confidence level and detect unseen disease classes limits the clinical implementation of artificial intelligence in the real world. We develop a foundation model with uncertainty estimation (FMUE) to detect 16 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieves a higher F1 score of 95.74% than other state-of-the-art algorithms (92.03%-93.66%) and improves to 97.44% with threshold strategy. The model achieves similar excellent performance on two external test sets from the same and different OCT machines. In human-model comparison, FMUE achieves a higher F1 score of 96.30% than retinal experts (86.95%, p = 0.004), senior doctors (82.71%, p < 0.001), junior doctors (66.55%, p < 0.001), and generative pretrained transformer 4 with vision (GPT-4V) (32.39%, p < 0.001). Besides, FMUE predicts high uncertainty scores for >85% images of non-target-category diseases or with low quality to prompt manual checks and prevent misdiagnosis. Our FMUE provides a trustworthy method for automatic retinal anomaly detection in a clinical open-set environment.</p>","PeriodicalId":9822,"journal":{"name":"Cell Reports Medicine","volume":" ","pages":"101876"},"PeriodicalIF":11.7000,"publicationDate":"2025-01-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11866418/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cell Reports Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.xcrm.2024.101876","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/12/19 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"CELL BIOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

Inability to express the confidence level and detect unseen disease classes limits the clinical implementation of artificial intelligence in the real world. We develop a foundation model with uncertainty estimation (FMUE) to detect 16 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieves a higher F1 score of 95.74% than other state-of-the-art algorithms (92.03%-93.66%) and improves to 97.44% with threshold strategy. The model achieves similar excellent performance on two external test sets from the same and different OCT machines. In human-model comparison, FMUE achieves a higher F1 score of 96.30% than retinal experts (86.95%, p = 0.004), senior doctors (82.71%, p < 0.001), junior doctors (66.55%, p < 0.001), and generative pretrained transformer 4 with vision (GPT-4V) (32.39%, p < 0.001). Besides, FMUE predicts high uncertainty scores for >85% images of non-target-category diseases or with low quality to prompt manual checks and prevent misdiagnosis. Our FMUE provides a trustworthy method for automatic retinal anomaly detection in a clinical open-set environment.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
提高人工智能的可靠性:基于光学相干层析的视网膜疾病诊断的不确定性估计基础模型。
无法表达置信度和检测未知疾病类别限制了人工智能在现实世界中的临床应用。我们建立了一个基于不确定性估计(FMUE)的基础模型,用于光学相干断层扫描(OCT)检测16种视网膜疾病。在内部测试集中,FMUE的F1得分为95.74%,高于其他最先进的算法(92.03%-93.66%),采用阈值策略后F1得分提高到97.44%。该模型在来自相同和不同OCT机器的两个外部测试集上取得了相似的优异性能。在人体模型对比中,FMUE的F1得分为96.30%,高于视网膜专家(86.95%,p = 0.004)和高级医生(82.71%,p = 85%)对非靶类疾病或质量较低的图像进行人工检查,防止误诊。我们的FMUE为临床开放集环境下的视网膜异常自动检测提供了一种可靠的方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Cell Reports Medicine
Cell Reports Medicine Biochemistry, Genetics and Molecular Biology-Biochemistry, Genetics and Molecular Biology (all)
CiteScore
15.00
自引率
1.40%
发文量
231
审稿时长
40 days
期刊介绍: Cell Reports Medicine is an esteemed open-access journal by Cell Press that publishes groundbreaking research in translational and clinical biomedical sciences, influencing human health and medicine. Our journal ensures wide visibility and accessibility, reaching scientists and clinicians across various medical disciplines. We publish original research that spans from intriguing human biology concepts to all aspects of clinical work. We encourage submissions that introduce innovative ideas, forging new paths in clinical research and practice. We also welcome studies that provide vital information, enhancing our understanding of current standards of care in diagnosis, treatment, and prognosis. This encompasses translational studies, clinical trials (including long-term follow-ups), genomics, biomarker discovery, and technological advancements that contribute to diagnostics, treatment, and healthcare. Additionally, studies based on vertebrate model organisms are within the scope of the journal, as long as they directly relate to human health and disease.
期刊最新文献
Regulation of LEAP2 by insulin and glucagon in mice and humans. Engineered human myogenic cells in hydrogels generate innervated vascularized myofibers within dystrophic mouse muscle on long-term engraftment. Neoadjuvant fuzuloparib combined with abiraterone for localized high-risk prostate cancer (FAST-PC): A single-arm phase 2 study. The gut microbiota-immune-brain axis: Therapeutic implications. A circadian and app-based personalized lighting intervention for the reduction of cancer-related fatigue.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1