多模态大语言模型解读儿科放射影像的能力。

IF 2.1 3区医学 Q2 PEDIATRICS Pediatric Radiology Pub Date : 2024-09-01 Epub Date: 2024-08-12 DOI:10.1007/s00247-024-06025-0

Thomas P Reith, Donna M D'Alessandro, Michael P D'Alessandro

{"title":"多模态大语言模型解读儿科放射影像的能力。","authors":"Thomas P Reith, Donna M D'Alessandro, Michael P D'Alessandro","doi":"10.1007/s00247-024-06025-0","DOIUrl":null,"url":null,"abstract":"Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images.Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.","PeriodicalId":19755,"journal":{"name":"Pediatric Radiology","volume":" ","pages":"1729-1737"},"PeriodicalIF":2.1000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Capability of multimodal large language models to interpret pediatric radiological images.\",\"authors\":\"Thomas P Reith, Donna M D'Alessandro, Michael P D'Alessandro\",\"doi\":\"10.1007/s00247-024-06025-0\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images.Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.\",\"PeriodicalId\":19755,\"journal\":{\"name\":\"Pediatric Radiology\",\"volume\":\" \",\"pages\":\"1729-1737\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2024-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Pediatric Radiology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s00247-024-06025-0\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/8/12 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q2\",\"JCRName\":\"PEDIATRICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pediatric Radiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00247-024-06025-0","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/12 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"PEDIATRICS","Score":null,"Total":0}

引用次数: 0

摘要

背景：专门针对儿科放射学的人工智能（AI）开发和研究十分匮乏。最新迭代的大型语言模型（LLMs），如 ChatGPT，除了能处理文本外，还能处理图像和视频输入。因此，理论上它们能够提供输入的放射图像的印象：评估多模态语言模型解读儿科放射影像的能力：收集并向 GPT-4（OpenAI，加利福尼亚州旧金山）、Gemini 1.5 Pro（谷歌，加利福尼亚州山景城）和 Claude 3 Opus（Anthropic，加利福尼亚州旧金山）提交了 30 个有医学意义的病例以及简短病史，共计 90 幅图像。人工智能反应由一名住院医师和一名主治医师记录并独立评估其准确性。95% 的置信区间采用调整 Wald 法确定：总体而言，模型正确诊断了 27.8% (25/90) 的图像（95% CI=19.5-37.8%），部分正确诊断了 13.3% (12/90) 的图像（95% CI=2.7-26.4%），错误诊断了 58.9% (53/90) 的图像（95% CI=48.6-68.5%）：结论：多模态 LLM 尚不能解读儿科放射影像。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

摘要图片

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

Capability of multimodal large language models to interpret pediatric radiological images.

Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.

Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images.

Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.

Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).

Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Pediatric Radiology 医学-核医学

CiteScore

4.40

自引率

17.40%

发文量

300

审稿时长

3-6 weeks

期刊介绍： Official Journal of the European Society of Pediatric Radiology, the Society for Pediatric Radiology and the Asian and Oceanic Society for Pediatric Radiology Pediatric Radiology informs its readers of new findings and progress in all areas of pediatric imaging and in related fields. This is achieved by a blend of original papers, complemented by reviews that set out the present state of knowledge in a particular area of the specialty or summarize specific topics in which discussion has led to clear conclusions. Advances in technology, methodology, apparatus and auxiliary equipment are presented, and modifications of standard techniques are described. Manuscripts submitted for publication must contain a statement to the effect that all human studies have been reviewed by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in an appropriate version of the 1964 Declaration of Helsinki. It should also be stated clearly in the text that all persons gave their informed consent prior to their inclusion in the study. Details that might disclose the identity of the subjects under study should be omitted.