C M Maron, S H Emile, N Horesh, M R Freund, G Pellino, S D Wexner
{"title":"Comparing answers of ChatGPT and Google Gemini to common questions on benign anal conditions.","authors":"C M Maron, S H Emile, N Horesh, M R Freund, G Pellino, S D Wexner","doi":"10.1007/s10151-024-03096-x","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>Chatbots have been increasingly used as a source of patient education. This study aimed to compare the answers of ChatGPT-4 and Google Gemini to common questions on benign anal conditions in terms of appropriateness, comprehensiveness, and language level.</p><p><strong>Methods: </strong>Each chatbot was asked a set of 30 questions on hemorrhoidal disease, anal fissures, and anal fistulas. The responses were assessed for appropriateness, comprehensiveness, and reference provision. The assessments were made by three subject experts who were unaware of the name of the chatbots. The language level of the chatbot answers was assessed using the Flesch-Kincaid Reading Ease score and grade level.</p><p><strong>Results: </strong>Overall, the answers provided by both models were appropriate and comprehensive. The answers of Google Gemini were more appropriate, comprehensive, and supported by references compared with the answers of ChatGPT. In addition, the agreement among the assessors on the appropriateness of Google Gemini answers was higher, attesting to a higher consistency. ChatGPT had a significantly higher Flesh-Kincaid grade level than Google Gemini (12.3 versus 10.6, p = 0.015), but a similar median Flesh-Kincaid Ease score.</p><p><strong>Conclusions: </strong>The answers of Google Gemini to questions on common benign anal conditions were more appropriate and comprehensive, and more often supported with references, than the answers of ChatGPT. The answers of both chatbots were at grade levels higher than the 6th grade level, which may be difficult for nonmedical individuals to comprehend.</p>","PeriodicalId":51192,"journal":{"name":"Techniques in Coloproctology","volume":"29 1","pages":"57"},"PeriodicalIF":2.9000,"publicationDate":"2025-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Techniques in Coloproctology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s10151-024-03096-x","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Introduction: Chatbots have been increasingly used as a source of patient education. This study aimed to compare the answers of ChatGPT-4 and Google Gemini to common questions on benign anal conditions in terms of appropriateness, comprehensiveness, and language level.
Methods: Each chatbot was asked a set of 30 questions on hemorrhoidal disease, anal fissures, and anal fistulas. The responses were assessed for appropriateness, comprehensiveness, and reference provision. The assessments were made by three subject experts who were unaware of the name of the chatbots. The language level of the chatbot answers was assessed using the Flesch-Kincaid Reading Ease score and grade level.
Results: Overall, the answers provided by both models were appropriate and comprehensive. The answers of Google Gemini were more appropriate, comprehensive, and supported by references compared with the answers of ChatGPT. In addition, the agreement among the assessors on the appropriateness of Google Gemini answers was higher, attesting to a higher consistency. ChatGPT had a significantly higher Flesh-Kincaid grade level than Google Gemini (12.3 versus 10.6, p = 0.015), but a similar median Flesh-Kincaid Ease score.
Conclusions: The answers of Google Gemini to questions on common benign anal conditions were more appropriate and comprehensive, and more often supported with references, than the answers of ChatGPT. The answers of both chatbots were at grade levels higher than the 6th grade level, which may be difficult for nonmedical individuals to comprehend.
期刊介绍：
Techniques in Coloproctology is an international journal fully devoted to diagnostic and operative procedures carried out in the management of colorectal diseases. Imaging, clinical physiology, laparoscopy, open abdominal surgery and proctoperineology are the main topics covered by the journal. Reviews, original articles, technical notes and short communications with many detailed illustrations render this publication indispensable for coloproctologists and related specialists. Both surgeons and gastroenterologists are represented on the distinguished Editorial Board, together with pathologists, radiologists and basic scientists from all over the world. The journal is strongly recommended to those who wish to be updated on recent developments in the field, and improve the standards of their work.
Manuscripts submitted for publication must contain a statement to the effect that all human studies have been reviewed by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in an appropriate version of the 1965 Declaration of Helsinki. It should also be stated clearly in the text that all persons gave their informed consent prior to their inclusion in the study. Details that might disclose the identity of the subjects under study should be omitted. Reports of animal experiments must state that the Principles of Laboratory Animal Care (NIH publication no. 86-23 revised 1985) were followed as were applicable national laws (e.g. the current version of the German Law on the Protection of Animals). The Editor-in-Chief reserves the right to reject manuscripts that do not comply with the above-mentioned requirements. Authors will be held responsible for false statements or for failure to fulfill such requirements.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
