{"title":"Assessing the Knowledge of ChatGPT in Answering Questions Regarding Female Urology.","authors":"Hakan Cakir, Ufuk Caglar, Ahmet Halis, Omer Sarilar, Huseyin Burak Yazili, Faruk Ozgor","doi":"10.22037/uj.v21i.8194","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>With the recent increase in the use of artificial intelligence in the medical field, this study aimed to evaluate the accuracy and adequacy of ChatGPT's responses to questions related to female urology.</p><p><strong>Methods: </strong>Intensive internet research was performed to prepare a frequently asked question (FAQs) list. Scientific questions were created in accordance with the European Urology Association (EAU) Non-neurogenic Female Lower Urinary Tract Symptoms Guidelines, EAU Chronic Pelvis Pain Guidelines, and EAU Neuro-Urology Guidelines. All answers by ChatGPT were analysed by two experienced urologists and each answer was scored between 1 and 4 by the physicians. A score of 1 was the highest and showed that the answer was completely true and sufficient. The reproducibility of ChatGPT answers was evaluated by asking each question twice using two different computers.</p><p><strong>Results: </strong>A total of 96 (97.0%) ChatGPT answers about female urology were accurate and sufficient, and categorized as grade 1. Additionally, two (2.0%) answers were scored as grade 2, and one answer (1.0%) was scored as grade 3. None of ChatGPT's responses about female urology were classified as grade 4. In total, 83 questions were prepared according to EAU guidelines recommendations, and ChatGPT gave complete accurate and satisfactory answers for 68 (82.9%) questions. The reproducibility rate was highest for ChatGPT answers for questions related to urinary incontinence, pelvic organ prolapses, and pelvic pain syndromes, and reproducibility rate was 100% for each subgroup. The reproducibility rate for ChatGPT answers was lowest for CPG questions (84.1%).</p><p><strong>Conclusion: </strong>For the first time our study revealed that ChatGPT had an excellent accuracy rate in answering questions related to female urology with 97% success rate. In addition, the outcomes of this study showed that ChatGPT accurately and satisfactorily answered 82.9% of questions about female urology based on EAU guidelines.</p>","PeriodicalId":23416,"journal":{"name":"Urology Journal","volume":" ","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2024-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Urology Journal","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.22037/uj.v21i.8194","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"UROLOGY & NEPHROLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose: With the recent increase in the use of artificial intelligence in the medical field, this study aimed to evaluate the accuracy and adequacy of ChatGPT's responses to questions related to female urology.
Methods: Intensive internet research was performed to prepare a frequently asked question (FAQs) list. Scientific questions were created in accordance with the European Urology Association (EAU) Non-neurogenic Female Lower Urinary Tract Symptoms Guidelines, EAU Chronic Pelvis Pain Guidelines, and EAU Neuro-Urology Guidelines. All answers by ChatGPT were analysed by two experienced urologists and each answer was scored between 1 and 4 by the physicians. A score of 1 was the highest and showed that the answer was completely true and sufficient. The reproducibility of ChatGPT answers was evaluated by asking each question twice using two different computers.
Results: A total of 96 (97.0%) ChatGPT answers about female urology were accurate and sufficient, and categorized as grade 1. Additionally, two (2.0%) answers were scored as grade 2, and one answer (1.0%) was scored as grade 3. None of ChatGPT's responses about female urology were classified as grade 4. In total, 83 questions were prepared according to EAU guidelines recommendations, and ChatGPT gave complete accurate and satisfactory answers for 68 (82.9%) questions. The reproducibility rate was highest for ChatGPT answers for questions related to urinary incontinence, pelvic organ prolapses, and pelvic pain syndromes, and reproducibility rate was 100% for each subgroup. The reproducibility rate for ChatGPT answers was lowest for CPG questions (84.1%).
Conclusion: For the first time our study revealed that ChatGPT had an excellent accuracy rate in answering questions related to female urology with 97% success rate. In addition, the outcomes of this study showed that ChatGPT accurately and satisfactorily answered 82.9% of questions about female urology based on EAU guidelines.
期刊介绍：
As the official journal of the Urology and Nephrology Research Center (UNRC) and the Iranian Urological Association (IUA), Urology Journal is a comprehensive digest of useful information on modern urology. Emphasis is on practical information that reflects the latest diagnostic and treatment techniques. Our objectives are to provide an exceptional source of current and clinically relevant research in the discipline of urology, to reflect the scientific work and progress of our colleagues, and to present the articles in a logical, timely, and concise format that meets the diverse needs of today’s urologist.
Urology Journal publishes manuscripts on urology and kidney transplantation, all of which undergo extensive peer review by recognized authorities in the field prior to their acceptance for publication. Accordingly, original articles, case reports, and letters to editor are encouraged.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
