首页 > 最新文献

Journal of Biomedical Informatics最新文献

英文 中文
Accelerating probabilistic privacy-preserving medical record linkage: A three-party MPC approach 加速概率隐私保护医疗记录链接:一种三方MPC方法
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-01 DOI: 10.1016/j.jbi.2025.104920
Şeyma Selcan Mağara, Noah Dietrich, Ali Burak Ünal, Mete Akgün

Objective:

Record linkage is essential for integrating data from multiple sources with diverse applications in real-world healthcare and research. Probabilistic Privacy-Preserving Record Linkage (PPRL) enables this integration occurs, while protecting sensitive information from unauthorized access, especially when datasets lack exact identifiers. As privacy regulations evolve and multi-institutional collaborations expand globally, there is a growing demand for methods that effectively balance security, accuracy, and efficiency. However, ensuring both privacy and scalability in large-scale record linkage remains a key challenge.

Method:

This paper presents a novel and efficient PPRL method based on a secure 3-party computation (MPC) framework. Our approach allows multiple parties to compute linkage results without exposing their private inputs and significantly improves the speed of linkage process compared to existing PPRL solutions.

Result:

Our method preserves the linkage quality of a state-of-the-art (SOTA) MPC-based PPRL method while achieving up to 14 times faster performance. For example, linking a record against a database of 10,000 records takes just 8.74 s in a realistic network with 700 Mbps bandwidth and 60 ms latency, compared to 92.32 s with the SOTA method. Even on a slower internet connection with 100 Mbps bandwidth and 60 ms latency, the linkage completes in 28 s, where as the SOTA method requires 287.96 s. These results demonstrate the significant scalability and efficiency improvements of our approach.

Conclusion:

Our novel PPRL method, based on secure 3-party computation, offers an efficient and scalable solution for large-scale record linkage while ensuring privacy protection. The approach demonstrates significant performance improvements, making it a promising tool for secure data integration in privacy-sensitive sectors.
目的:记录链接对于在现实世界的医疗保健和研究中整合来自多个来源的不同应用的数据至关重要。概率隐私保护记录链接(PPRL)实现了这种集成,同时保护敏感信息免受未经授权的访问,特别是当数据集缺乏精确的标识符时。随着隐私法规的发展和多机构合作在全球范围内的扩展,对有效平衡安全性、准确性和效率的方法的需求不断增长。然而,在大规模记录链接中,确保隐私和可扩展性仍然是一个关键的挑战。方法:提出一种基于安全三方计算(MPC)框架的新型高效PPRL方法。我们的方法允许多方在不暴露其私有输入的情况下计算链接结果,与现有的PPRL解决方案相比,显著提高了链接过程的速度。结果:我们的方法保留了最先进的(SOTA)基于mpc的PPRL方法的连接质量,同时实现了高达14倍的性能提升。例如,在带宽为700 Mbps、延迟为60 ms的实际网络中,将一条记录与包含10,000条记录的数据库相关联只需要8.74 s,而使用SOTA方法需要92.32 s。即使在带宽为100mbps、延迟为60ms的较慢的互联网连接上,连接也需要在28秒内完成,而SOTA方法需要287.96秒。这些结果表明,我们的方法具有显著的可扩展性和效率改进。结论:基于安全三方计算的PPRL方法在保证隐私保护的同时,为大规模记录链接提供了高效、可扩展的解决方案。该方法显示了显著的性能改进,使其成为隐私敏感领域中安全数据集成的有前途的工具。
{"title":"Accelerating probabilistic privacy-preserving medical record linkage: A three-party MPC approach","authors":"Şeyma Selcan Mağara,&nbsp;Noah Dietrich,&nbsp;Ali Burak Ünal,&nbsp;Mete Akgün","doi":"10.1016/j.jbi.2025.104920","DOIUrl":"10.1016/j.jbi.2025.104920","url":null,"abstract":"<div><h3>Objective:</h3><div>Record linkage is essential for integrating data from multiple sources with diverse applications in real-world healthcare and research. Probabilistic Privacy-Preserving Record Linkage (PPRL) enables this integration occurs, while protecting sensitive information from unauthorized access, especially when datasets lack exact identifiers. As privacy regulations evolve and multi-institutional collaborations expand globally, there is a growing demand for methods that effectively balance security, accuracy, and efficiency. However, ensuring both privacy and scalability in large-scale record linkage remains a key challenge.</div></div><div><h3>Method:</h3><div>This paper presents a novel and efficient PPRL method based on a secure 3-party computation (MPC) framework. Our approach allows multiple parties to compute linkage results without exposing their private inputs and significantly improves the speed of linkage process compared to existing PPRL solutions.</div></div><div><h3>Result:</h3><div>Our method preserves the linkage quality of a state-of-the-art (SOTA) MPC-based PPRL method while achieving up to 14 times faster performance. For example, linking a record against a database of 10,000 records takes just 8.74 s in a realistic network with 700 Mbps bandwidth and 60 ms latency, compared to 92.32 s with the SOTA method. Even on a slower internet connection with 100 Mbps bandwidth and 60 ms latency, the linkage completes in 28 s, where as the SOTA method requires 287.96 s. These results demonstrate the significant scalability and efficiency improvements of our approach.</div></div><div><h3>Conclusion:</h3><div>Our novel PPRL method, based on secure 3-party computation, offers an efficient and scalable solution for large-scale record linkage while ensuring privacy protection. The approach demonstrates significant performance improvements, making it a promising tool for secure data integration in privacy-sensitive sectors.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104920"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145223419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Definitions to data flow: Operationalizing MIABIS in HL7 FHIR 数据流的定义:在HL7 FHIR中实现MIABIS。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-09-27 DOI: 10.1016/j.jbi.2025.104919
Radovan Tomášik , Šimon Koňár , Niina Eklund , Cäcilia Engels , Zdenka Dudova , Radoslava Kacová , Roman Hrstka , Petr Holub

Objective

Biobanks and biomolecular resources are increasingly central to data-driven biomedical research, encompassing not only metadata but also granular, sample-related data from diverse sources such as healthcare systems, national registries, and research outputs. However, the lack of a standardised, machine-readable format for representing such data limits interoperability, data reuse and integration into clinical and research environments. While MIABIS provides a conceptual model for biobank data, its abstract nature and reliance on heterogeneous implementations create barriers to practical, scalable adoption. This study presents a pragmatic, operational implementation of MIABIS focused on enabling real-world exchange and integration of sample-level data.

Methods

We systematically evaluated established data exchange standards, comparing HL7 FHIR and OMOP CDM with respect to their suitability for structuring sample-related data in a semantically robust and machine-readable form. Based on this analysis, we developed a FHIR-based representation of MIABIS that supports complex biobank structures and enables integration with federated data infrastructures. Supporting tools, including a Python library and an implementation guide, were created to ensure usability across diverse research and clinical contexts.

Results

We created nine interoperable FHIR profiles covering core MIABIS entities, ensuring consistency with FHIR standards. To support adoption, we developed an open-source Python library that abstracts FHIR interactions and provides schema validation for MIABIS-compliant data. The library was integrated into an ETL tool in operation at Czech Node of BBMRI-ERIC, European Biobanking and Biomolecular Resources Research Infrastructure, to demonstrate usability with real-world sample-related data. Separately, we validated the representation of MIABIS entities at the organisational level by converting the data structures of BBMRI-ERIC Directory into FHIR, demonstrating compatibility with federated data infrastructures.

Conclusion

This work delivers a machine-readable, interoperable implementation of MIABIS, enabling the exchange of both organisational and sample-level data across biobanks and health information systems. By integrating MIABIS with HL7 FHIR, we provide a host of reusable tools and mechanisms for further evolution of the data model. Combined, these benefits can help with the integration into clinical and research workflows, supporting data discoverability, reuse, and cross-institutional collaboration in biomedical research.
目的:生物银行和生物分子资源在数据驱动的生物医学研究中越来越重要,不仅包括元数据,还包括来自不同来源(如医疗保健系统、国家登记处和研究成果)的颗粒状样本相关数据。然而,缺乏一种标准化的、机器可读的格式来表示这些数据,限制了互操作性、数据重用和临床和研究环境的集成。虽然MIABIS为生物银行数据提供了一个概念模型,但它的抽象性和对异构实现的依赖为实际的、可扩展的采用创造了障碍。本研究提出了一种实用的、可操作的MIABIS实现方法,重点是实现样本级数据的真实交换和集成。方法:我们系统地评估了已建立的数据交换标准,比较了HL7 FHIR和OMOP CDM在以语义鲁棒性和机器可读形式构建样本相关数据方面的适用性。基于这一分析,我们开发了一个基于fhir的MIABIS表示,它支持复杂的生物库结构,并能够与联邦数据基础设施集成。包括Python库和实现指南在内的支持工具被创建,以确保在不同的研究和临床环境中可用性。结果:我们创建了9个可互操作的FHIR配置文件,涵盖了核心MIABIS实体,确保了与FHIR标准的一致性。为了支持采用,我们开发了一个开源Python库,它抽象了FHIR交互,并为符合miabis的数据提供了模式验证。该库被整合到BBMRI-ERIC捷克节点的ETL工具中,欧洲生物银行和生物分子资源研究基础设施,以展示与现实世界样本相关数据的可用性。另外,我们通过将BBMRI-ERIC目录的数据结构转换为FHIR,验证了MIABIS实体在组织级别的表示,展示了与联邦数据基础设施的兼容性。结论:这项工作提供了一个机器可读、可互操作的MIABIS实现,使生物库和卫生信息系统之间的组织和样本级数据交换成为可能。通过将MIABIS与HL7 FHIR集成,我们为数据模型的进一步发展提供了大量可重用的工具和机制。综合起来,这些优势可以帮助整合到临床和研究工作流程中,支持生物医学研究中的数据发现、重用和跨机构协作。
{"title":"Definitions to data flow: Operationalizing MIABIS in HL7 FHIR","authors":"Radovan Tomášik ,&nbsp;Šimon Koňár ,&nbsp;Niina Eklund ,&nbsp;Cäcilia Engels ,&nbsp;Zdenka Dudova ,&nbsp;Radoslava Kacová ,&nbsp;Roman Hrstka ,&nbsp;Petr Holub","doi":"10.1016/j.jbi.2025.104919","DOIUrl":"10.1016/j.jbi.2025.104919","url":null,"abstract":"<div><h3>Objective</h3><div>Biobanks and biomolecular resources are increasingly central to data-driven biomedical research, encompassing not only metadata but also granular, sample-related data from diverse sources such as healthcare systems, national registries, and research outputs. However, the lack of a standardised, machine-readable format for representing such data limits interoperability, data reuse and integration into clinical and research environments. While MIABIS provides a conceptual model for biobank data, its abstract nature and reliance on heterogeneous implementations create barriers to practical, scalable adoption. This study presents a pragmatic, operational implementation of MIABIS focused on enabling real-world exchange and integration of sample-level data.</div></div><div><h3>Methods</h3><div>We systematically evaluated established data exchange standards, comparing HL7 FHIR and OMOP CDM with respect to their suitability for structuring sample-related data in a semantically robust and machine-readable form. Based on this analysis, we developed a FHIR-based representation of MIABIS that supports complex biobank structures and enables integration with federated data infrastructures. Supporting tools, including a Python library and an implementation guide, were created to ensure usability across diverse research and clinical contexts.</div></div><div><h3>Results</h3><div>We <em>created nine interoperable FHIR profiles</em> covering core MIABIS entities, ensuring consistency with FHIR standards. To support adoption, we <em>developed an open-source Python library</em> that abstracts FHIR interactions and provides schema validation for MIABIS-compliant data. The <em>library was integrated into an ETL tool</em> in operation at Czech Node of BBMRI-ERIC, European Biobanking and Biomolecular Resources Research Infrastructure, to demonstrate usability with real-world sample-related data. Separately, we validated the representation of MIABIS entities at the organisational level by converting the data structures of BBMRI-ERIC Directory into FHIR, demonstrating compatibility with federated data infrastructures.</div></div><div><h3>Conclusion</h3><div>This work delivers a machine-readable, interoperable implementation of MIABIS, enabling the exchange of both organisational and sample-level data across biobanks and health information systems. By integrating MIABIS with HL7 FHIR, we provide a host of reusable tools and mechanisms for further evolution of the data model. Combined, these benefits can help with the integration into clinical and research workflows, supporting data discoverability, reuse, and cross-institutional collaboration in biomedical research.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104919"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145191636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A REDCap advanced randomization module to meet the needs of modern trials 一个REDCap高级随机化模块,以满足现代试验的需要。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-04 DOI: 10.1016/j.jbi.2025.104925
Luke Stevens , Nan Kennedy , Rob J. Taylor , Adam Lewis , Frank E. Harrell Jr , Matthew S. Shotwell , Emily S. Serdoz , Gordon R. Bernard , Wesley H. Self , Christopher J. Lindsell , Paul A. Harris , Jonathan D. Casey

Objective

Since 2012, the electronic data capture platform REDCap has included an embedded randomization module allowing a single randomization per study record with the ability to stratify by variables such as study site and participant sex at birth. In recent years, platform, adaptive, decentralized, and pragmatic trials have gained popularity. These trial designs often require approaches to randomization not supported by the original REDCap randomization module, including randomizing patients into multiple domains or at multiple points in time, changing allocation tables to add or drop study groups, or adaptively changing allocation ratios based on data from previously enrolled participants. Our team aimed to develop new randomization functions to address these issues.

Methods

A collaborative process facilitated by the NIH-funded Trial Innovation Network was initiated to modernize the randomization module in REDCap, incorporating feedback from clinical trialists, biostatisticians, technologists, and other experts.

Results

This effort led to the development of an advanced randomization module within the REDCap platform. In addition to supporting platform, adaptive, decentralized, and pragmatic trials, the new module introduces several new features, such as improved support for blinded randomization, additional randomization metadata capture (e.g., user identity and timestamp), additional tools allowing REDCap administrators to support investigators using the randomization module, and the ability for clinicians participating in pragmatic or decentralized trials to perform randomization through a survey without needing log-in access to the study database. As of June 19, 2025, multiple randomizations have been used in 211 projects from 55 institutions, randomizations with real-time trigger logic in 108 projects from 64 institutions, and blinded group allocation in 24 projects from 17 institutions.

Conclusion

The new randomization module aims to streamline the randomization process, improve trial efficiency, and ensure robust data integrity, thereby supporting the conduct of more sophisticated and adaptive clinical trials.
目的:自2012年以来,电子数据采集平台REDCap包含了一个嵌入式随机化模块,允许每个研究记录进行单个随机化,并能够根据研究地点和参与者出生时的性别等变量进行分层。近年来,平台化、自适应、去中心化、实用化的审判越来越受欢迎。这些试验设计通常需要采用原始REDCap随机化模块不支持的随机化方法,包括将患者随机分配到多个领域或多个时间点,改变分配表以增加或减少研究组,或根据先前入组的参与者的数据自适应地改变分配比例。我们的团队旨在开发新的随机化功能来解决这些问题。方法:在美国国立卫生研究院资助的试验创新网络的推动下,启动了一个协作过程,将临床试验学家、生物统计学家、技术专家和其他专家的反馈结合起来,使REDCap中的随机化模块现代化。结果:这一努力促成了REDCap平台内高级随机化模块的开发。除了支持平台、自适应、去中心化和实用的试验之外,新模块还引入了几个新功能,例如改进了对盲法随机化的支持、额外的随机化元数据捕获(例如,用户身份和时间戳)、允许REDCap管理员使用随机化模块支持调查人员的额外工具。参与实用或分散试验的临床医生无需登录研究数据库即可通过调查执行随机化的能力。截至2025年6月19日,55所院校211个项目采用了多重随机化,64所院校108个项目采用了实时触发逻辑随机化,17所院校24个项目采用了盲法分组。结论:新的随机化模块旨在简化随机化过程,提高试验效率,确保数据的完整性,从而支持开展更复杂和适应性更强的临床试验。
{"title":"A REDCap advanced randomization module to meet the needs of modern trials","authors":"Luke Stevens ,&nbsp;Nan Kennedy ,&nbsp;Rob J. Taylor ,&nbsp;Adam Lewis ,&nbsp;Frank E. Harrell Jr ,&nbsp;Matthew S. Shotwell ,&nbsp;Emily S. Serdoz ,&nbsp;Gordon R. Bernard ,&nbsp;Wesley H. Self ,&nbsp;Christopher J. Lindsell ,&nbsp;Paul A. Harris ,&nbsp;Jonathan D. Casey","doi":"10.1016/j.jbi.2025.104925","DOIUrl":"10.1016/j.jbi.2025.104925","url":null,"abstract":"<div><h3>Objective</h3><div>Since 2012, the electronic data capture platform REDCap has included an embedded randomization module allowing a single randomization per study record with the ability to stratify by variables such as study site and participant sex at birth. In recent years, platform, adaptive, decentralized, and pragmatic trials have gained popularity. These trial designs often require approaches to randomization not supported by the original REDCap randomization module, including randomizing patients into multiple domains or at multiple points in time, changing allocation tables to add or drop study groups, or adaptively changing allocation ratios based on data from previously enrolled participants. Our team aimed to develop new randomization functions to address these issues.</div></div><div><h3>Methods</h3><div>A collaborative process facilitated by the NIH-funded Trial Innovation Network was initiated to modernize the randomization module in REDCap, incorporating feedback from clinical trialists, biostatisticians, technologists, and other experts.</div></div><div><h3>Results</h3><div>This effort led to the development of an advanced randomization module within the REDCap platform. In addition to supporting platform, adaptive, decentralized, and pragmatic trials, the new module introduces several new features, such as improved support for blinded randomization, additional randomization metadata capture (e.g., user identity and timestamp), additional tools allowing REDCap administrators to support investigators using the randomization module, and the ability for clinicians participating in pragmatic or decentralized trials to perform randomization through a survey without needing log-in access to the study database. As of June 19, 2025, multiple randomizations have been used in 211 projects from 55 institutions, randomizations with real-time trigger logic in 108 projects from 64 institutions, and blinded group allocation in 24 projects from 17 institutions.</div></div><div><h3>Conclusion</h3><div>The new randomization module aims to streamline the randomization process, improve trial efficiency, and ensure robust data integrity, thereby supporting the conduct of more sophisticated and adaptive clinical trials.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104925"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145238683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring multimodal large language models on transthoracic Echocardiogram (TTE) tasks for cardiovascular decision support 探索多模态大语言模型在经胸超声心动图(TTE)任务中的心血管决策支持。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-23 DOI: 10.1016/j.jbi.2025.104930
Jianfu Li , Yiming Li , Zenan Sun , Evan Yu , Ahmed M. Abdelhameed , Weiguo Cao , Haifang Li , Jianping He , Pengze Li , Jingna Feng , Yue Yu , Xinyue Hu , Manqi Li , Rakesh Kumar , Yifang Dang , Fang Li , Shahyar M Gharacholou , Cui Tao

Objective

Multimodal large language models (LLMs) offer new potential for enhancing cardiovascular decision support, particularly in interpreting echocardiographic data. This study systematically evaluates and benchmarks foundation models from diverse domains on echocardiogram-based tasks to assess their effectiveness, limitations and potential in clinical cardiovascular applications.

Methods

We curated three cardiovascular imaging datasets—EchoNet-Dynamic, TMED2, and an expert-annotated echocardiogram (TTE) dataset—to evaluate performance on four critical tasks: (1) cardiac function evaluation through ejection fraction (EF) prediction, (2) cardiac view classification, (3) aortic stenosis (AS) severity assessment, and (4) cardiovascular disease classification. We evaluated six multimodal LLMs: EchoClip (cardiovascular-specific), BiomedGPT and LLaVA-Med (medical-domain), and MiniCPM-V 2.6, LLaMA-3-Vision-Alpha, and Gemini-1.5 (general-domain). Models were assessed using zero-shot, few-shot, and fine-tuning strategies, where applicable. Performance was measured using mean absolute error (MAE) and root mean squared error (RMSE) for EF prediction, and accuracy, precision, recall, and F1 score for classification tasks.

Results

Domain-specific models such as EchoClip demonstrated the strongest zero-shot performance in EF prediction, achieving an MAE of 10.34. General-domain models showed limited effectiveness without adaptation, with MiniCPM-V 2.6 reporting an MAE of 251.92. Fine-tuning significantly improved outcomes; for example, MiniCPM-V 2.6′s MAE decreased to 31.93, and view classification accuracy increased from 20 % to 63.05 %. In classification tasks, EchoClip achieved F1 scores of 0.2716 for AS severity and 0.4919 for disease classification but exhibited limited performance in view classification (F1 = 0.1457). Few-shot learning yielded modest gains but was generally less effective than fine-tuning.

Conclusions

This evaluation and benchmarking study demonstrated the importance of domain-specific pretraining and model adaptation in cardiovascular decision support tasks. Cardiovascular-focused models and fine-tuned general-domain models achieved superior performance, especially for complex assessments such as EF estimation. These findings offer critical insights into the current capabilities and future directions for clinically meaningful AI integration in cardiovascular medicine.
目的:多模态大语言模型(LLMs)为增强心血管决策支持提供了新的潜力,特别是在解释超声心动图数据方面。本研究系统地评估和基准了基于超声心动图任务的不同领域的基础模型,以评估其在临床心血管应用中的有效性、局限性和潜力。方法:我们整理了三个心血管成像数据集——echonet - dynamic、TMED2和专家注释的超声心动图(TTE)数据集,以评估四个关键任务的表现:(1)通过射血分数(EF)预测心功能评估,(2)心脏视图分类,(3)主动脉狭窄(AS)严重程度评估,以及(4)心血管疾病分类。我们评估了6种多模式llm: EchoClip(心血管特异性)、BiomedGPT和LLaVA-Med(医学领域),以及MiniCPM-V 2.6、LLaMA-3-Vision-Alpha和Gemini-1.5(通用领域)。在适用的情况下,使用零射击、少射击和微调策略评估模型。使用EF预测的平均绝对误差(MAE)和均方根误差(RMSE)以及分类任务的准确度、精密度、召回率和F1分数来衡量性能。结果:EchoClip等领域特定模型在EF预测中表现出最强的零射击性能,MAE为10.34。通用领域模型在没有适应的情况下显示出有限的有效性,minicpm - v2.6报告的MAE为251.92。微调显著改善了结果;例如,minicpm - v2.6的MAE下降到31.93,视图分类准确率从20 %提高到63.05 %。在分类任务中,EchoClip在AS严重程度和疾病分类方面的F1得分分别为0.2716和0.4919,但在视图分类方面表现有限(F1 = 0.1457)。几次学习产生了适度的收益,但通常不如微调有效。结论:这项评估和基准研究证明了特定领域的预训练和模型适应在心血管决策支持任务中的重要性。以心血管为中心的模型和精细调整的通用领域模型取得了卓越的性能,特别是对于复杂的评估,如EF估计。这些发现为心血管医学中有临床意义的人工智能整合的当前能力和未来方向提供了重要见解。
{"title":"Exploring multimodal large language models on transthoracic Echocardiogram (TTE) tasks for cardiovascular decision support","authors":"Jianfu Li ,&nbsp;Yiming Li ,&nbsp;Zenan Sun ,&nbsp;Evan Yu ,&nbsp;Ahmed M. Abdelhameed ,&nbsp;Weiguo Cao ,&nbsp;Haifang Li ,&nbsp;Jianping He ,&nbsp;Pengze Li ,&nbsp;Jingna Feng ,&nbsp;Yue Yu ,&nbsp;Xinyue Hu ,&nbsp;Manqi Li ,&nbsp;Rakesh Kumar ,&nbsp;Yifang Dang ,&nbsp;Fang Li ,&nbsp;Shahyar M Gharacholou ,&nbsp;Cui Tao","doi":"10.1016/j.jbi.2025.104930","DOIUrl":"10.1016/j.jbi.2025.104930","url":null,"abstract":"<div><h3>Objective</h3><div>Multimodal large language models (LLMs) offer new potential for enhancing cardiovascular decision support, particularly in interpreting echocardiographic data. This study systematically evaluates and benchmarks foundation models from diverse domains on echocardiogram-based tasks to assess their effectiveness, limitations and potential in clinical cardiovascular applications.</div></div><div><h3>Methods</h3><div>We curated three cardiovascular imaging datasets—EchoNet-Dynamic, TMED2, and an expert-annotated echocardiogram (TTE) dataset—to evaluate performance on four critical tasks: (1) cardiac function evaluation through ejection fraction (EF) prediction, (2) cardiac view classification, (3) aortic stenosis (AS) severity assessment, and (4) cardiovascular disease classification. We evaluated six multimodal LLMs: EchoClip (cardiovascular-specific), BiomedGPT and LLaVA-Med (medical-domain), and MiniCPM-V 2.6, LLaMA-3-Vision-Alpha, and Gemini-1.5 (general-domain). Models were assessed using zero-shot, few-shot, and fine-tuning strategies, where applicable. Performance was measured using mean absolute error (MAE) and root mean squared error (RMSE) for EF prediction, and accuracy, precision, recall, and F1 score for classification tasks.</div></div><div><h3>Results</h3><div>Domain-specific models such as EchoClip demonstrated the strongest zero-shot performance in EF prediction, achieving an MAE of 10.34. General-domain models showed limited effectiveness without adaptation, with MiniCPM-V 2.6 reporting an MAE of 251.92. Fine-tuning significantly improved outcomes; for example, MiniCPM-V 2.6′s MAE decreased to 31.93, and view classification accuracy increased from 20 % to 63.05 %. In classification tasks, EchoClip achieved F1 scores of 0.2716 for AS severity and 0.4919 for disease classification but exhibited limited performance in view classification (F1 = 0.1457). Few-shot learning yielded modest gains but was generally less effective than fine-tuning.</div></div><div><h3>Conclusions</h3><div>This evaluation and benchmarking study demonstrated the importance of domain-specific pretraining and model adaptation in cardiovascular decision support tasks. Cardiovascular-focused models and fine-tuned general-domain models achieved superior performance, especially for complex assessments such as EF estimation. These findings offer critical insights into the current capabilities and future directions for clinically meaningful AI integration in cardiovascular medicine.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104930"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145370273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MF-DTA: Predicting drug–target affinity with multi-modal feature fusion model MF-DTA:用多模态特征融合模型预测药物-靶标亲和力。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-10 DOI: 10.1016/j.jbi.2025.104926
Yanlei Kang , Haoyu Zhuang , Yunliang Jiang , Zhong Li
The prediction of drug–target interactions (DTIs) and binding affinities (DTAs) plays a pivotal role in drug discovery and design. However, most existing methods fail to fully exploit the rich multimodal information inherent in molecular structures. In this study, we propose a multimodal feature fusion model, MF-DTA. On the representational level, MF-DTA introduces the molecular fragment graph, generated via BRICS-based decomposition, as a novel modality. This representation enables a more intuitive capture of the structural characteristics and pharmacophore-related information of drug molecules. In terms of model architecture, a deformable convolutional layer is applied for the protein residue–residue contact map (hereafter referred to as contact map) to flexibly adjust the distribution of sampling points and enhance the representational capability. To effectively integrate the multimodal information from both drug and target branches, a mixture-of-experts (MoE)-based multihead attention mechanism is employed for local fusion, while a dual-decoder architecture facilitates cross-modal interaction between drug and target features. The final output yields a high-quality prediction of binding affinity. Cross-validation experiments conducted on several benchmark datasets demonstrate that MF-DTA consistently outperforms state-of-the-art methods. Specifically, it achieves CI improvements of 0.1%, 0.5%, and 0.3% over the best-performing baseline models in the Davis, KIBA and BindingDB datasets, respectively, and exceeds traditional models by 1% to 2% on average. The model also ranks among the best performers in terms of the MSE and Rm 2 metrics. Model visualization further supports its interpretability, confirming that it successfully learns meaningful drug–target interaction patterns.To further assess the practical utility of the proposed model, we apply it to screen potential candidate compounds from a natural product library targeting tubulin. In summary, MF-DTA offers not only accurate and robust binding affinity prediction capabilities but also strong interpretability, making it a powerful and practical tool for drug design and target identification.
药物-靶标相互作用(DTIs)和结合亲和力(DTAs)的预测在药物发现和设计中起着关键作用。然而,大多数现有的方法都不能充分利用分子结构中固有的丰富的多模态信息。在这项研究中,我们提出了一个多模态特征融合模型,MF-DTA。在表征层面上,MF-DTA引入了通过基于金砖四国的分解生成的分子片段图,作为一种新的模态。这种表示可以更直观地捕获药物分子的结构特征和药团相关信息。在模型架构上,对蛋白质残馀-残馀接触图(以下简称接触图)采用可变形卷积层,灵活调整采样点分布,增强表征能力。为了有效地整合来自药物和靶标分支的多模态信息,采用基于混合专家(MoE)的多头注意机制进行局部融合,采用双解码器架构实现药物和靶标特征之间的跨模态交互。最后的输出产生了高质量的结合亲和力预测。在几个基准数据集上进行的交叉验证实验表明,MF-DTA始终优于最先进的方法。具体来说,它比Davis、KIBA和BindingDB数据集中表现最好的基线模型分别提高了0.1%、0.5%和0.3%,平均比传统模型高出1%到2%。该模型在MSE和Rm2指标方面也名列前茅。模型可视化进一步支持其可解释性,确认它成功地学习了有意义的药物-靶标相互作用模式。为了进一步评估所提出的模型的实际效用,我们将其应用于筛选针对微管蛋白的天然产物库中的潜在候选化合物。综上所述,MF-DTA不仅具有准确、稳健的结合亲和力预测能力,而且具有很强的可解释性,使其成为药物设计和靶点鉴定的强大实用工具。
{"title":"MF-DTA: Predicting drug–target affinity with multi-modal feature fusion model","authors":"Yanlei Kang ,&nbsp;Haoyu Zhuang ,&nbsp;Yunliang Jiang ,&nbsp;Zhong Li","doi":"10.1016/j.jbi.2025.104926","DOIUrl":"10.1016/j.jbi.2025.104926","url":null,"abstract":"<div><div>The prediction of drug–target interactions (DTIs) and binding affinities (DTAs) plays a pivotal role in drug discovery and design. However, most existing methods fail to fully exploit the rich multimodal information inherent in molecular structures. In this study, we propose a multimodal feature fusion model, MF-DTA. On the representational level, MF-DTA introduces the molecular fragment graph, generated via BRICS-based decomposition, as a novel modality. This representation enables a more intuitive capture of the structural characteristics and pharmacophore-related information of drug molecules. In terms of model architecture, a deformable convolutional layer is applied for the protein residue–residue contact map (hereafter referred to as contact map) to flexibly adjust the distribution of sampling points and enhance the representational capability. To effectively integrate the multimodal information from both drug and target branches, a mixture-of-experts (MoE)-based multihead attention mechanism is employed for local fusion, while a dual-decoder architecture facilitates cross-modal interaction between drug and target features. The final output yields a high-quality prediction of binding affinity. Cross-validation experiments conducted on several benchmark datasets demonstrate that MF-DTA consistently outperforms state-of-the-art methods. Specifically, it achieves CI improvements of 0.1%, 0.5%, and 0.3% over the best-performing baseline models in the Davis, KIBA and BindingDB datasets, respectively, and exceeds traditional models by 1% to 2% on average. The model also ranks among the best performers in terms of the MSE and R<sub>m</sub> <span><math><msup><mrow></mrow><mrow><mn>2</mn></mrow></msup></math></span> metrics. Model visualization further supports its interpretability, confirming that it successfully learns meaningful drug–target interaction patterns.To further assess the practical utility of the proposed model, we apply it to screen potential candidate compounds from a natural product library targeting tubulin. In summary, MF-DTA offers not only accurate and robust binding affinity prediction capabilities but also strong interpretability, making it a powerful and practical tool for drug design and target identification.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104926"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145280336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Soft label-guided transformer for radiology report generation 用于放射学报告生成的软标签引导变压器。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-13 DOI: 10.1016/j.jbi.2025.104924
Xinyao Liu , Junchang Xin , Qi Shen , Zhihong Huang , Zhiqiong Wang

Objective:

Radiology report provides important references for physicians’ treatment decisions by including descriptions and diagnostic results of imaging. Automatic generation of radiology report reduces the workload of physicians and significantly improves work efficiency. However, the existing report generation methods use image-text conversion to generate reports directly from medical images, and fail to fully simulate the radiologist’s diagnostic process of “examine first, describe later”. Therefore, existing methods often can only generate general normal descriptions, and it is difficult to accurately describe the specific lesion features.

Methods:

To address this issue, we mimic the working mode of radiologists by first checking whether the patient suffers from a certain disease, and then using the learned medical knowledge to describe the images to form a report. We propose a soft label-guided transformer (SLGT) for radiology report generation. Firstly, the pseudo-labels of the samples are obtained, and the soft label-guided attention mechanism is utilized to highlight features related to the disease labels in the encoding stage. Secondly, text features from the decoding phase and image features are aligned, and the generated text features are used to guide the potential representations. Finally, a hybrid loss is designed that includes losses for text generation, disease classification, and visual-textual alignment. Optimization of SLGT using the hybrid loss allows the model to learn richer features that are more relevant to disease abnormalities, which improves the performance of the model.

Results:

The proposed SLGT is evaluated on the widely used IU X-ray, MIMIC-CXR, and COV-CTR datasets. The experiments show that the proposed model SLGT outperforms the previous state-of-the-art models on three datasets.

Conclusion:

This work improves the performance of automatically generating medical reports, making their application in computer-aided diagnosis feasible.
目的:影像学报告包括影像学描述和诊断结果,为医生的治疗决策提供重要参考。自动生成放射科报告,减少了医生的工作量,显著提高了工作效率。然而,现有的报告生成方法采用图像-文本转换,直接从医学图像生成报告,不能完全模拟放射科医生“先检查后描述”的诊断过程。因此,现有的方法往往只能生成一般的正常描述,难以准确描述病变的具体特征。方法:为了解决这一问题,我们模仿放射科医生的工作模式,首先检查患者是否患有某种疾病,然后利用所学的医学知识对图像进行描述,形成报告。我们提出了一种软标签引导变压器(SLGT)用于放射学报告生成。首先获取样本的伪标签,利用软标签引导注意机制在编码阶段突出与疾病标签相关的特征;其次,将解码阶段的文本特征与图像特征对齐,利用生成的文本特征指导潜在表征;最后,设计了一个混合损失,包括文本生成、疾病分类和视觉文本对齐的损失。使用混合损失对SLGT进行优化,使模型能够学习与疾病异常更相关的更丰富的特征,从而提高模型的性能。结果:所提出的SLGT在广泛使用的IU X-ray、MIMIC-CXR和COV-CTR数据集上进行了评估。实验表明,本文提出的模型在三个数据集上的性能优于现有的最先进模型。结论:本工作提高了病历自动生成的性能,使病历在计算机辅助诊断中的应用成为可能。
{"title":"Soft label-guided transformer for radiology report generation","authors":"Xinyao Liu ,&nbsp;Junchang Xin ,&nbsp;Qi Shen ,&nbsp;Zhihong Huang ,&nbsp;Zhiqiong Wang","doi":"10.1016/j.jbi.2025.104924","DOIUrl":"10.1016/j.jbi.2025.104924","url":null,"abstract":"<div><h3>Objective:</h3><div>Radiology report provides important references for physicians’ treatment decisions by including descriptions and diagnostic results of imaging. Automatic generation of radiology report reduces the workload of physicians and significantly improves work efficiency. However, the existing report generation methods use image-text conversion to generate reports directly from medical images, and fail to fully simulate the radiologist’s diagnostic process of “examine first, describe later”. Therefore, existing methods often can only generate general normal descriptions, and it is difficult to accurately describe the specific lesion features.</div></div><div><h3>Methods:</h3><div>To address this issue, we mimic the working mode of radiologists by first checking whether the patient suffers from a certain disease, and then using the learned medical knowledge to describe the images to form a report. We propose a soft label-guided transformer (SLGT) for radiology report generation. Firstly, the pseudo-labels of the samples are obtained, and the soft label-guided attention mechanism is utilized to highlight features related to the disease labels in the encoding stage. Secondly, text features from the decoding phase and image features are aligned, and the generated text features are used to guide the potential representations. Finally, a hybrid loss is designed that includes losses for text generation, disease classification, and visual-textual alignment. Optimization of SLGT using the hybrid loss allows the model to learn richer features that are more relevant to disease abnormalities, which improves the performance of the model.</div></div><div><h3>Results:</h3><div>The proposed SLGT is evaluated on the widely used IU X-ray, MIMIC-CXR, and COV-CTR datasets. The experiments show that the proposed model SLGT outperforms the previous state-of-the-art models on three datasets.</div></div><div><h3>Conclusion:</h3><div>This work improves the performance of automatically generating medical reports, making their application in computer-aided diagnosis feasible.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104924"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145300755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
ADENER: A syntax-augmented grid-tagging model for Adverse Drug Event extraction in social media 社交媒体中不良药物事件提取的语法增强网格标记模型。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-22 DOI: 10.1016/j.jbi.2025.104944
Weiru Fu , Hao Li , Ling Luo, Hongfei Lin

Objective:

Adverse Drug Event (ADE) extraction from social media is a critical yet challenging task due to the semantic similarity between adverse effects and therapeutic indications, as well as the prevalence of overlapping and discontinuous mentions often caused by comorbid conditions. This study aims to develop a robust model for accurate ADE extraction from noisy and irregular social media texts.

Methods:

We propose ADENER, a grid-tagging architecture that models ADE extraction as multi-label word-pair classification. ADENER incorporates two core encoding mechanisms: the convolutional capture layer fuses multi-dimensional textual features, captures long-range word-pair dependencies via dilated convolutions, and enhances interactions through semantic association matrices for social media text irregularities; the syntactic affine layer integrates path-level dependency information to enhance global logic understanding, enabling the model to distinguish between therapeutic symptom entities and ADE entities through syntactic cues. The decoding stage uses four-type relational labels to uniformly decode flat, overlapping, and discontinuous ADE mentions.

Results:

We evaluated ADENER on three widely used ADE extraction datasets: CADEC, CADECv2, SMM4H. The model achieved F1 scores of 74.64%, 77.97%, 61.73% on these datasets, respectively, outperforming all compared baseline models while maintaining competitive computational efficiency. The results demonstrate the effectiveness of our model in addressing the challenges posed by irregular and noisy social media data.

Conclusion:

ADENER offers a unified and effective solution for ADE extraction from social media, capable of handling flat, overlapping, and discontinuous entity mentions and correctly distinguishing ADE entities from therapeutic symptom entities. By incorporating convolutional capture layers for semantic word-pair interactions and syntactic affine layers for dependency-based logic understanding, our approach significantly improves extraction accuracy, providing a valuable tool for pharmacovigilance research and real-world drug safety monitoring.
目的:由于不良反应和治疗指征之间的语义相似性,以及通常由合并症引起的重叠和不连续提及的普遍存在,从社交媒体中提取药物不良事件(ADE)是一项关键但具有挑战性的任务。本研究旨在开发一个强大的模型,用于从嘈杂和不规则的社交媒体文本中准确提取ADE。方法:我们提出了ADENER,这是一种网格标记架构,将ADE提取建模为多标签词对分类。ADENER包含两种核心编码机制:卷积捕获层融合多维文本特征,通过扩展卷积捕获远程词对依赖关系,并通过语义关联矩阵增强社交媒体文本不规则性的交互;句法仿射层整合路径级依赖信息,增强整体逻辑理解,使模型能够通过句法线索区分治疗症状实体和ADE实体。解码阶段使用四种类型的关系标签来统一解码平坦的、重叠的和不连续的ADE提及。结果:我们在CADEC、CADECv2、SMM4H三个广泛使用的ADE提取数据集上对ADENER进行了评估。该模型在这些数据集上的F1得分分别为74.64%、77.97%和61.73%,在保持有竞争力的计算效率的同时,优于所有比较基线模型。结果表明,我们的模型在应对不规则和嘈杂的社交媒体数据带来的挑战方面是有效的。结论:ADENER为社交媒体的ADE提取提供了统一有效的解决方案,能够处理平坦、重叠、不连续的实体提及,并正确区分ADE实体与治疗症状实体。通过结合语义词对交互的卷积捕获层和基于依赖的逻辑理解的句法仿射层,我们的方法显着提高了提取精度,为药物警戒研究和现实世界的药物安全监测提供了有价值的工具。
{"title":"ADENER: A syntax-augmented grid-tagging model for Adverse Drug Event extraction in social media","authors":"Weiru Fu ,&nbsp;Hao Li ,&nbsp;Ling Luo,&nbsp;Hongfei Lin","doi":"10.1016/j.jbi.2025.104944","DOIUrl":"10.1016/j.jbi.2025.104944","url":null,"abstract":"<div><h3>Objective:</h3><div>Adverse Drug Event (ADE) extraction from social media is a critical yet challenging task due to the semantic similarity between adverse effects and therapeutic indications, as well as the prevalence of overlapping and discontinuous mentions often caused by comorbid conditions. This study aims to develop a robust model for accurate ADE extraction from noisy and irregular social media texts.</div></div><div><h3>Methods:</h3><div>We propose ADENER, a grid-tagging architecture that models ADE extraction as multi-label word-pair classification. ADENER incorporates two core encoding mechanisms: the convolutional capture layer fuses multi-dimensional textual features, captures long-range word-pair dependencies via dilated convolutions, and enhances interactions through semantic association matrices for social media text irregularities; the syntactic affine layer integrates path-level dependency information to enhance global logic understanding, enabling the model to distinguish between therapeutic symptom entities and ADE entities through syntactic cues. The decoding stage uses four-type relational labels to uniformly decode flat, overlapping, and discontinuous ADE mentions.</div></div><div><h3>Results:</h3><div>We evaluated ADENER on three widely used ADE extraction datasets: CADEC, CADECv2, SMM4H. The model achieved F1 scores of 74.64%, 77.97%, 61.73% on these datasets, respectively, outperforming all compared baseline models while maintaining competitive computational efficiency. The results demonstrate the effectiveness of our model in addressing the challenges posed by irregular and noisy social media data.</div></div><div><h3>Conclusion:</h3><div>ADENER offers a unified and effective solution for ADE extraction from social media, capable of handling flat, overlapping, and discontinuous entity mentions and correctly distinguishing ADE entities from therapeutic symptom entities. By incorporating convolutional capture layers for semantic word-pair interactions and syntactic affine layers for dependency-based logic understanding, our approach significantly improves extraction accuracy, providing a valuable tool for pharmacovigilance research and real-world drug safety monitoring.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104944"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145355020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Synthetic-to-real attentive deep learning for Alzheimer’s assessment: A domain-agnostic framework for ROCF scoring 用于阿尔茨海默氏症评估的综合到真实的专注深度学习:ROCF评分的领域不可知框架。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-17 DOI: 10.1016/j.jbi.2025.104929
Kassem Anis Bouali, Elena Šikudová

Objective:

Early diagnosis of Alzheimer’s disease depends on accessible cognitive assessments, such as the Rey-Osterrieth Complex Figure (ROCF) test. However, manual scoring of this test is labor-intensive and subjective, which introduces experimental biases. Additionally, deep learning models face challenges due to the limited availability of annotated clinical data, particularly for assessments like the ROCF test. This scarcity of data restricts model generalization and exacerbates domain shifts across different populations.

Methods:

We propose a novel framework comprising a data synthesis pipeline and ROCF-Net, a deep learning model specifically designed for ROCF scoring. The synthesis pipeline is lightweight and capable of generating realistic, diverse, and annotated ROCF drawings. ROCF-Net, on the other hand, is a cross-domain scoring model engineered to address domain discrepancies in stroke texture and line artifacts. It maintains high scoring accuracy through a novel line-specific attention mechanism tailored to the unique characteristics of ROCF drawings.

Results:

Unlike conventional synthetic medical imaging methods, our approach generates ROCF drawings that accurately reflect Alzheimer’s-specific abnormalities with minimal computational cost. Our scoring model achieves SOTA performance across differently sourced datasets, with a Mean Absolute Error (MAE) of 3.53 and a Pearson Correlation Coefficient (PCC) of 0.86. This demonstrates both high predictive accuracy and computational efficiency, outperforming existing ROCF scoring methods that rely on Convolutional Neural Networks (CNNs) while avoiding the overhead of parameter-heavy transformer models. We also show that training on our synthetic data generalizes as well as training on real clinical data, where the difference in performance was minimal (MAE differed by 1.43 and PCC by 0.07), indicating no statistically significant performance gap.

Conclusion:

Our work introduces four contributions: (1) a cost-effective pipeline for generating synthetic ROCF data, reducing dependency on clinical datasets; (2) a domain-agnostic model for automated ROCF scoring across diverse drawing styles; (3) a lightweight attention mechanism aligning model decisions with clinical scoring for transparency; and (4) a bias-aware framework using synthetic data to reduce demographic disparities, promoting fair cognitive assessment across populations.
目的:阿尔茨海默病的早期诊断依赖于可获得的认知评估,如Rey-Osterrieth Complex Figure (ROCF)测试。然而,手工评分是劳动密集型和主观的,这引入了实验偏差。此外,由于标注临床数据的可用性有限,深度学习模型面临挑战,特别是对于像ROCF测试这样的评估。这种数据的稀缺性限制了模型的泛化,并加剧了不同人群之间的领域转移。方法:我们提出了一个新的框架,包括一个数据合成管道和ROCF- net,一个专门为ROCF评分设计的深度学习模型。合成管道是轻量级的,能够生成真实的、多样的、带注释的ROCF图纸。另一方面,ROCF-Net是一个跨域评分模型,用于解决笔画纹理和线条伪像中的域差异。它通过针对ROCF图纸的独特特征量身定制的新颖的线特定注意机制保持高评分精度。结果:与传统的合成医学成像方法不同,我们的方法以最小的计算成本生成准确反映阿尔茨海默病特异性异常的ROCF图。我们的评分模型在不同来源的数据集上实现了SOTA性能,平均绝对误差(MAE)为3.53,Pearson相关系数(PCC)为0.86。这证明了高预测精度和计算效率,优于现有的依赖卷积神经网络(cnn)的ROCF评分方法,同时避免了重参数变压器模型的开销。我们还表明,在我们的合成数据上的训练与在真实临床数据上的训练一样一般化,其中性能差异很小(MAE差1.43,PCC差0.07),表明没有统计学上显著的性能差距。结论:我们的工作引入了四个贡献:(1)成本效益高的管道生成合成ROCF数据,减少对临床数据集的依赖;(2)跨不同画风的自动ROCF评分的领域不可知模型;(3)将模型决策与临床透明度评分相结合的轻量级注意机制;(4)利用综合数据构建偏见感知框架,减少人口差异,促进人群间的公平认知评估。
{"title":"Synthetic-to-real attentive deep learning for Alzheimer’s assessment: A domain-agnostic framework for ROCF scoring","authors":"Kassem Anis Bouali,&nbsp;Elena Šikudová","doi":"10.1016/j.jbi.2025.104929","DOIUrl":"10.1016/j.jbi.2025.104929","url":null,"abstract":"<div><h3>Objective:</h3><div>Early diagnosis of Alzheimer’s disease depends on accessible cognitive assessments, such as the Rey-Osterrieth Complex Figure (ROCF) test. However, manual scoring of this test is labor-intensive and subjective, which introduces experimental biases. Additionally, deep learning models face challenges due to the limited availability of annotated clinical data, particularly for assessments like the ROCF test. This scarcity of data restricts model generalization and exacerbates domain shifts across different populations.</div></div><div><h3>Methods:</h3><div>We propose a novel framework comprising a data synthesis pipeline and ROCF-Net, a deep learning model specifically designed for ROCF scoring. The synthesis pipeline is lightweight and capable of generating realistic, diverse, and annotated ROCF drawings. ROCF-Net, on the other hand, is a cross-domain scoring model engineered to address domain discrepancies in stroke texture and line artifacts. It maintains high scoring accuracy through a novel line-specific attention mechanism tailored to the unique characteristics of ROCF drawings.</div></div><div><h3>Results:</h3><div>Unlike conventional synthetic medical imaging methods, our approach generates ROCF drawings that accurately reflect Alzheimer’s-specific abnormalities with minimal computational cost. Our scoring model achieves SOTA performance across differently sourced datasets, with a Mean Absolute Error (MAE) of 3.53 and a Pearson Correlation Coefficient (PCC) of 0.86. This demonstrates both high predictive accuracy and computational efficiency, outperforming existing ROCF scoring methods that rely on Convolutional Neural Networks (CNNs) while avoiding the overhead of parameter-heavy transformer models. We also show that training on our synthetic data generalizes as well as training on real clinical data, where the difference in performance was minimal (MAE differed by 1.43 and PCC by 0.07), indicating no statistically significant performance gap.</div></div><div><h3>Conclusion:</h3><div>Our work introduces four contributions: (1) a cost-effective pipeline for generating synthetic ROCF data, reducing dependency on clinical datasets; (2) a domain-agnostic model for automated ROCF scoring across diverse drawing styles; (3) a lightweight attention mechanism aligning model decisions with clinical scoring for transparency; and (4) a bias-aware framework using synthetic data to reduce demographic disparities, promoting fair cognitive assessment across populations.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104929"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145329252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Multi-feature machine learning for enhanced drug–drug interaction prediction 多特征机器学习增强药物-药物相互作用预测。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-08 DOI: 10.1016/j.jbi.2025.104923
Qiuyang Feng , Xiao Huang
Drug–drug interactions are a major concern in healthcare, as concurrent drug use can cause severe adverse effects. Existing machine learning methods often neglect data imbalance and DDI directionality, limiting clinical reliability. To overcome these issues, we employed GPT-4o Large Language Model to convert free-text DDI descriptions into structured triplets for directionality analysis and applied SMOTE to alleviate class imbalance. Using four key drug features (molecular fingerprints, enzymes, pathways, targets), our Deep Neural Networks (DNN) achieved 88.9% accuracy and showed an average AUPR gain of 0.68 for minority classes attributable to SMOTE. By applying attention-based feature importance analysis, we demonstrated that the most influential feature in the DNN model was supported by pharmacological evidence. These results demonstrate the effectiveness of our framework for accurate and robust DDI prediction. The source code and data are available at https://github.com/FrankFengF/Drug-drug-interaction-prediction-
药物-药物相互作用是医疗保健中的一个主要问题,因为同时使用药物会导致严重的不良反应。现有的机器学习方法往往忽略了数据的不平衡和DDI的方向性,限制了临床的可靠性。为了克服这些问题,我们使用gpt - 40大型语言模型将自由文本DDI描述转换为结构化三元组进行方向性分析,并应用SMOTE来缓解类不平衡。利用四个关键的药物特征(分子指纹图谱、酶、途径、靶标),我们的深度神经网络(DNN)达到了88.9%的准确率,并且显示出归因于SMOTE的少数类别的平均AUPR增益为0.68。通过应用基于注意的特征重要性分析,我们证明DNN模型中最具影响力的特征得到了药理学证据的支持。这些结果证明了我们的框架对准确和稳健的DDI预测的有效性。源代码和数据可从https://github.com/FrankFengF/Drug-drug-interaction-prediction获得。
{"title":"Multi-feature machine learning for enhanced drug–drug interaction prediction","authors":"Qiuyang Feng ,&nbsp;Xiao Huang","doi":"10.1016/j.jbi.2025.104923","DOIUrl":"10.1016/j.jbi.2025.104923","url":null,"abstract":"<div><div>Drug–drug interactions are a major concern in healthcare, as concurrent drug use can cause severe adverse effects. Existing machine learning methods often neglect data imbalance and DDI directionality, limiting clinical reliability. To overcome these issues, we employed GPT-4o Large Language Model to convert free-text DDI descriptions into structured triplets for directionality analysis and applied SMOTE to alleviate class imbalance. Using four key drug features (molecular fingerprints, enzymes, pathways, targets), our Deep Neural Networks (DNN) achieved 88.9% accuracy and showed an average AUPR gain of 0.68 for minority classes attributable to SMOTE. By applying attention-based feature importance analysis, we demonstrated that the most influential feature in the DNN model was supported by pharmacological evidence. These results demonstrate the effectiveness of our framework for accurate and robust DDI prediction. The source code and data are available at <span><span>https://github.com/FrankFengF/Drug-drug-interaction-prediction-</span><svg><path></path></svg></span></div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104923"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145258203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
From image to report: automating lung cancer screening interpretation and reporting with vision-language models 从图像到报告:使用视觉语言模型自动化肺癌筛查解释和报告。
IF 4.5 2区 医学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2025-11-01 Epub Date: 2025-10-11 DOI: 10.1016/j.jbi.2025.104931
Tien-Yu Chang , Qinglin Gou , Leyi Zhao , Tiancheng Zhou , Hongyu Chen , Dong Yang , Huiwen Ju , Kaleb E. Smith , Chengkun Sun , Jinqian Pan , Yu Huang , Xing He , Xuhong Zhang , Daguang Xu , Jie Xu , Jiang Bian , Aokun Chen

Objective

Lung cancer is the most prevalent cancer and the leading cause of cancer-related death in the United States. Lung cancer screening with low-dose computed tomography (LDCT) helps identify lung cancer at an early stage and thus improves overall survival. The growing adoption of LDCT screening has increased radiologists’ workload and demands specialized training to accurately interpret LDCT images and report findings. Advances in artificial intelligence (AI), including large language models (LLMs) and vision models, could help reduce this burden and improve accuracy.

Methods

We devised LUMEN (Lung cancer screening with Unified Multimodal Evaluation and Navigation), a multimodal AI framework that mimics the radiologist’s workflow by identifying nodules in LDCT images, generating their characteristics, and drafting corresponding radiology reports in accordance with reporting guidelines. LUMEN integrates computer vision, vision-language models (VLMs), and LLMs. To assess our system, we developed a benchmarking framework to evaluate the lung cancer screening reports generated based on the findings and management criteria outlined in the Lung Imaging Reporting and Data System (Lung-RADS). It extracts them from radiology reports and measures clinical accuracy—focusing on information that is clinically important for lung cancer screening—independently of report format.

Results

This complement exists LLM/VLM in semantic accuracy metrics and provides a more comprehensive view of system performance. Our lung cancer screening report generation system achieved unparalleled performance compared to contemporary VLM systems, including M3D CT2Report. Furthermore, compared to standard LLM metrics, the clinical metrics we designed for lung cancer screening more accurately reflect the clinical utility of the generated reports.

Conclusion

LUMEN demonstrates the feasibility of generating clinically accurate lung nodule reports from LDCT images through a nodule-centric VQA approach, highlighting the potential of integrating VLMs and LLMs to support radiologists in lung cancer screening workflows. Our findings also underscore the importance of applying clinically meaningful evaluation metrics in developing medical AI systems.
目的:肺癌是美国最常见的癌症,也是癌症相关死亡的主要原因。肺癌筛查低剂量计算机断层扫描(LDCT)有助于在早期发现肺癌,从而提高总体生存率。越来越多地采用LDCT筛查增加了放射科医生的工作量,需要专门的培训来准确地解释LDCT图像并报告结果。人工智能(AI)的进步,包括大型语言模型(llm)和视觉模型,可以帮助减轻这种负担并提高准确性。方法:我们设计了LUMEN(肺癌筛查与统一多模式评估和导航),这是一个多模式人工智能框架,通过识别LDCT图像中的结节,生成其特征,并根据报告指南起草相应的放射学报告,模拟放射科医生的工作流程。LUMEN集成了计算机视觉、视觉语言模型(vlm)和llm。为了评估我们的系统,我们制定了一个基准框架来评估基于肺成像报告和数据系统(lung - rads)中概述的发现和管理标准生成的肺癌筛查报告。它从放射学报告中提取数据,并测量临床准确性——专注于对肺癌筛查具有临床重要性的信息——独立于报告格式。结果:语义准确性度量中存在LLM/VLM的补充,并提供了更全面的系统性能视图。我们的肺癌筛查报告生成系统与当代VLM系统(包括M3D, CT2Report和MedM3DVLM)相比具有无与伦比的性能。此外,与标准LLM指标相比,我们为肺癌筛查设计的临床指标更准确地反映了生成报告的临床效用。结论:LUMEN证明了通过以结节为中心的VQA方法从LDCT图像生成临床准确的肺结节报告的可行性,突出了整合vlm和llm以支持放射科医生肺癌筛查工作流程的潜力。我们的研究结果还强调了在开发医疗人工智能系统中应用临床有意义的评估指标的重要性。
{"title":"From image to report: automating lung cancer screening interpretation and reporting with vision-language models","authors":"Tien-Yu Chang ,&nbsp;Qinglin Gou ,&nbsp;Leyi Zhao ,&nbsp;Tiancheng Zhou ,&nbsp;Hongyu Chen ,&nbsp;Dong Yang ,&nbsp;Huiwen Ju ,&nbsp;Kaleb E. Smith ,&nbsp;Chengkun Sun ,&nbsp;Jinqian Pan ,&nbsp;Yu Huang ,&nbsp;Xing He ,&nbsp;Xuhong Zhang ,&nbsp;Daguang Xu ,&nbsp;Jie Xu ,&nbsp;Jiang Bian ,&nbsp;Aokun Chen","doi":"10.1016/j.jbi.2025.104931","DOIUrl":"10.1016/j.jbi.2025.104931","url":null,"abstract":"<div><h3>Objective</h3><div>Lung cancer is the most prevalent cancer and the leading cause of cancer-related death in the United States. Lung cancer screening with low-dose computed tomography (LDCT) helps identify lung cancer at an early stage and thus improves overall survival. The growing adoption of LDCT screening has increased radiologists’ workload and demands specialized training to accurately interpret LDCT images and report findings. Advances in artificial intelligence (AI), including large language models (LLMs) and vision models, could help reduce this burden and improve accuracy.</div></div><div><h3>Methods</h3><div>We devised LUMEN (Lung cancer screening with Unified Multimodal Evaluation and Navigation), a multimodal AI framework that mimics the radiologist’s workflow by identifying nodules in LDCT images, generating their characteristics, and drafting corresponding radiology reports in accordance with reporting guidelines. LUMEN integrates computer vision, vision-language models (VLMs), and LLMs. To assess our system, we developed a benchmarking framework to evaluate the lung cancer screening reports generated based on the findings and management criteria outlined in the Lung Imaging Reporting and Data System (Lung-RADS). It extracts them from radiology reports and measures clinical accuracy—focusing on information that is clinically important for lung cancer screening—independently of report format.</div></div><div><h3>Results</h3><div>This complement exists LLM/VLM in semantic accuracy metrics and provides a more comprehensive view of system performance. Our lung cancer screening report generation system achieved unparalleled performance compared to contemporary VLM systems, including M3D CT2Report. Furthermore, compared to standard LLM metrics, the clinical metrics we designed for lung cancer screening more accurately reflect the clinical utility of the generated reports.</div></div><div><h3>Conclusion</h3><div>LUMEN demonstrates the feasibility of generating clinically accurate lung nodule reports from LDCT images through a nodule-centric VQA approach, highlighting the potential of integrating VLMs and LLMs to support radiologists in lung cancer screening workflows. Our findings also underscore the importance of applying clinically meaningful evaluation metrics in developing medical AI systems.</div></div>","PeriodicalId":15263,"journal":{"name":"Journal of Biomedical Informatics","volume":"171 ","pages":"Article 104931"},"PeriodicalIF":4.5,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145286192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Journal of Biomedical Informatics
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1