ChatGPT as a medical education resource in cardiology: Mitigating replicability challenges and optimizing model performance

IF 3 3区 医学 Q2 CARDIAC & CARDIOVASCULAR SYSTEMS Current Problems in Cardiology Pub Date : 2024-10-10 DOI:10.1016/j.cpcardiol.2024.102879
{"title":"ChatGPT as a medical education resource in cardiology: Mitigating replicability challenges and optimizing model performance","authors":"","doi":"10.1016/j.cpcardiol.2024.102879","DOIUrl":null,"url":null,"abstract":"<div><div>Given the rapid development of large language models (LLMs), such as ChatGPT, in its ability to understand and generate human-like texts, these technologies inspired efforts to explore their capabilities in natural language processing tasks, especially those in healthcare contexts. The performance of these tools have been evaluated thoroughly across medicine in diverse tasks, including standardized medical examinations, medical-decision making, and many others. In this journal, Anaya et al. published a study comparing the readability metrics of medical education resources formulated by ChatGPT with those of major U.S. institutions (AHA, ACC, HFSA) about heart failure. In this work, we provide a critical review of this article and further describe approaches to help mitigate challenges in reproducibility of studies evaluating LLMs in cardiology. Additionally, we provide suggestions to optimize sampling of responses provided by LLMs for future studies. Overall, while the study by Anaya et al. provides a meaningful contribution to literature of LLMs in cardiology, further comprehensive studies are necessary to address current limitations and further strengthen our understanding of these novel tools.</div></div>","PeriodicalId":51006,"journal":{"name":"Current Problems in Cardiology","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Current Problems in Cardiology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0146280624005140","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CARDIAC & CARDIOVASCULAR SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Given the rapid development of large language models (LLMs), such as ChatGPT, in its ability to understand and generate human-like texts, these technologies inspired efforts to explore their capabilities in natural language processing tasks, especially those in healthcare contexts. The performance of these tools have been evaluated thoroughly across medicine in diverse tasks, including standardized medical examinations, medical-decision making, and many others. In this journal, Anaya et al. published a study comparing the readability metrics of medical education resources formulated by ChatGPT with those of major U.S. institutions (AHA, ACC, HFSA) about heart failure. In this work, we provide a critical review of this article and further describe approaches to help mitigate challenges in reproducibility of studies evaluating LLMs in cardiology. Additionally, we provide suggestions to optimize sampling of responses provided by LLMs for future studies. Overall, while the study by Anaya et al. provides a meaningful contribution to literature of LLMs in cardiology, further comprehensive studies are necessary to address current limitations and further strengthen our understanding of these novel tools.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
作为心脏病学医学教育资源的 ChatGPT:减轻可复制性挑战,优化模型性能。
鉴于大型语言模型(LLM)(如 ChatGPT)在理解和生成类人文本方面的快速发展,这些技术激发了人们探索其在自然语言处理任务中的能力,尤其是在医疗保健领域。这些工具的性能已在医学领域的各种任务中进行了全面评估,包括标准化医学检查、医疗决策等。Anaya 等人在该期刊上发表了一项研究,比较了 ChatGPT 与美国主要机构(AHA、ACC、HFSA)制定的有关心力衰竭的医学教育资源的可读性指标。在这项工作中,我们对这篇文章进行了批判性评论,并进一步介绍了有助于减轻心脏病学 LLM 评估研究可重复性挑战的方法。此外,我们还为未来的研究提供了优化 LLM 所提供回答的取样建议。总之,虽然 Anaya 等人的研究为心脏病学中的 LLMs 文献做出了有意义的贡献,但仍有必要开展进一步的综合研究,以解决当前的局限性,并进一步加强我们对这些新型工具的了解。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Current Problems in Cardiology
Current Problems in Cardiology 医学-心血管系统
CiteScore
4.80
自引率
2.40%
发文量
392
审稿时长
6 days
期刊介绍: Under the editorial leadership of noted cardiologist Dr. Hector O. Ventura, Current Problems in Cardiology provides focused, comprehensive coverage of important clinical topics in cardiology. Each monthly issues, addresses a selected clinical problem or condition, including pathophysiology, invasive and noninvasive diagnosis, drug therapy, surgical management, and rehabilitation; or explores the clinical applications of a diagnostic modality or a particular category of drugs. Critical commentary from the distinguished editorial board accompanies each monograph, providing readers with additional insights. An extensive bibliography in each issue saves hours of library research.
期刊最新文献
Effectiveness of Emotional-Focused Coping on Heart-Focused Anxiety in Patients Prior to Cardiac Catheterization. Unveiling the link between physical parameters and safety in cardiac rehabilitation: Longitudinal observational study: Physical parameters and cardiac adverse events. Heart transplantation in juvenile-onset systemic sclerosis with primary cardiac involvement: report of two cases and comprehensive literature review Editorial Board Table of Contents
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1