Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT-Omni and Gemini-Pro Across Repeated Intervals with Case Scenario and Open-Ended Queries.

IF 2 4区 医学 Q3 ONCOLOGY Oncology Research and Treatment Pub Date : 2025-03-12 DOI:10.1159/000545231
Seckin Tuna Kaplan
{"title":"Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT-Omni and Gemini-Pro Across Repeated Intervals with Case Scenario and Open-Ended Queries.","authors":"Seckin Tuna Kaplan","doi":"10.1159/000545231","DOIUrl":null,"url":null,"abstract":"<p><p>supporting clinical decision-making, diagnosis, and treatment. The study aims to compare the performance of ChatGPT-4o (Omni) and Gemini-pro in answering clinical questions and case scenarios related to gynecological oncology and to assess the consistency of their long-term responses.</p><p><strong>Methods: </strong>A two-phase comparative analysis was conducted. 700 clinical questions (350 per model) were developed and categorized into open-ended and case-scenario questions. Three months later, the same set of questions was presented again to evaluate any changes in performance for accuracy, completeness, and guideline adherence.</p><p><strong>Results: </strong>Omni outperformed Gemini-pro across all question types (p=0.001). Omni achieved a mean score of 5.9 for the basic open-ended questions, higher than Gemini, which had 5.1 (p=0.001). It also maintained a clear advantage in complex, open-ended questions, scoring a mean of 5.6 than Gemini AI's 4.2 (p=0.001). Omni scored a mean of 5.7 for basic case scenarios, while Gemini AI lagged with a mean score of 5 (p=0.001). Omni showed a modest improvement in complex, open-ended queries, with an increase of 0.2 points (+3.57%) (p=0.001). Omni provided more accurate and comprehensive responses in guideline adherence than Gemini, particularly in complex cases requiring nuanced judgment and adherence to oncology protocols. Its responses aligned with the latest guidelines, including the American Society of Clinical Oncology and the National Comprehensive Cancer Network.</p><p><strong>Conclusions: </strong>Omni is a more reliable and consistent model for answering questions related to gynecological cancers than Gemini. The stability of Omni's performance over time highlights its potential as an effective tool for clinical applications requiring high accuracy and consistency.</p>","PeriodicalId":19543,"journal":{"name":"Oncology Research and Treatment","volume":" ","pages":"1-10"},"PeriodicalIF":2.0000,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Oncology Research and Treatment","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1159/000545231","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

supporting clinical decision-making, diagnosis, and treatment. The study aims to compare the performance of ChatGPT-4o (Omni) and Gemini-pro in answering clinical questions and case scenarios related to gynecological oncology and to assess the consistency of their long-term responses.

Methods: A two-phase comparative analysis was conducted. 700 clinical questions (350 per model) were developed and categorized into open-ended and case-scenario questions. Three months later, the same set of questions was presented again to evaluate any changes in performance for accuracy, completeness, and guideline adherence.

Results: Omni outperformed Gemini-pro across all question types (p=0.001). Omni achieved a mean score of 5.9 for the basic open-ended questions, higher than Gemini, which had 5.1 (p=0.001). It also maintained a clear advantage in complex, open-ended questions, scoring a mean of 5.6 than Gemini AI's 4.2 (p=0.001). Omni scored a mean of 5.7 for basic case scenarios, while Gemini AI lagged with a mean score of 5 (p=0.001). Omni showed a modest improvement in complex, open-ended queries, with an increase of 0.2 points (+3.57%) (p=0.001). Omni provided more accurate and comprehensive responses in guideline adherence than Gemini, particularly in complex cases requiring nuanced judgment and adherence to oncology protocols. Its responses aligned with the latest guidelines, including the American Society of Clinical Oncology and the National Comprehensive Cancer Network.

Conclusions: Omni is a more reliable and consistent model for answering questions related to gynecological cancers than Gemini. The stability of Omni's performance over time highlights its potential as an effective tool for clinical applications requiring high accuracy and consistency.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
3.20
自引率
0.00%
发文量
84
期刊介绍: With the first issue in 2014, the journal ''Onkologie'' has changed its title to ''Oncology Research and Treatment''. By this change, publisher and editor set the scene for the further development of this interdisciplinary journal. The English title makes it clear that the articles are published in English – a logical step for the journal, which is listed in all relevant international databases. For excellent manuscripts, a ''Fast Track'' was introduced: The review is carried out within 2 weeks; after acceptance the papers are published online within 14 days and immediately released as ''Editor’s Choice'' to provide the authors with maximum visibility of their results. Interesting case reports are published in the section ''Novel Insights from Clinical Practice'' which clearly highlights the scientific advances which the report presents.
期刊最新文献
Highlights of Translational and Molecular Research Presented at the European Society for Medical Oncology Annual Meeting 2024. Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT-Omni and Gemini-Pro Across Repeated Intervals with Case Scenario and Open-Ended Queries. Effect of Early Tumor Shrinkage and Depth of Response on the Clinical Outcomes of Patients with Unresectable Hepatocellular Carcinoma Treated with Transcatheter Arterial Chemoembolization and Lenvatinib Plus PD-1 Inhibitors. Improving Accuracy and Source Transparency in Responses to Soft Tissue Sarcoma Queries using GPT-4o Enhanced with German Evidence-Based Guidelines. Modifications to prostate cancer diagnosis following COVID-19 and following models.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1