Juan Bernardo Villarreal-Espinosa , Rodrigo Saad Berreta , Felicitas Allende , José Rafael Garcia , Salvador Ayala , Filippo Familiari , Jorge Chahla
{"title":"对有关前十字韧带手术常见问题的 ChatGPT 回答进行准确性评估。","authors":"Juan Bernardo Villarreal-Espinosa , Rodrigo Saad Berreta , Felicitas Allende , José Rafael Garcia , Salvador Ayala , Filippo Familiari , Jorge Chahla","doi":"10.1016/j.knee.2024.08.014","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><p>The emergence of artificial intelligence (AI) has allowed users to have access to large sources of information in a chat-like manner. Thereby, we sought to evaluate ChatGPT-4 response’s accuracy to the 10 patient most frequently asked questions (FAQs) regarding anterior cruciate ligament (ACL) surgery.</p></div><div><h3>Methods</h3><p>A list of the top 10 FAQs pertaining to ACL surgery was created after conducting a search through all Sports Medicine Fellowship Institutions listed on the Arthroscopy Association of North America (AANA) and American Orthopaedic Society of Sports Medicine (AOSSM) websites. A Likert scale was used to grade response accuracy by two sports medicine fellowship-trained surgeons. Cohen’s kappa was used to assess inter-rater agreement. Reproducibility of the responses over time was also assessed.</p></div><div><h3>Results</h3><p>Five of the 10 responses received a ‘completely accurate’ grade by two-fellowship trained surgeons with three additional replies receiving a ‘completely accurate’ status by at least one. Moreover, inter-rater reliability accuracy assessment revealed a moderate agreement between fellowship-trained attending physicians (weighted kappa = 0.57, 95% confidence interval 0.15–0.99). Additionally, 80% of the responses were reproducible over time.</p></div><div><h3>Conclusion</h3><p>ChatGPT can be considered an accurate additional tool to answer general patient questions regarding ACL surgery. None the less, patient–surgeon interaction should not be deferred and must continue to be the driving force for information retrieval. Thus, the general recommendation is to address any questions in the presence of a qualified specialist.</p></div>","PeriodicalId":56110,"journal":{"name":"Knee","volume":"51 ","pages":"Pages 84-92"},"PeriodicalIF":1.6000,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Accuracy assessment of ChatGPT responses to frequently asked questions regarding anterior cruciate ligament surgery\",\"authors\":\"Juan Bernardo Villarreal-Espinosa , Rodrigo Saad Berreta , Felicitas Allende , José Rafael Garcia , Salvador Ayala , Filippo Familiari , Jorge Chahla\",\"doi\":\"10.1016/j.knee.2024.08.014\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><p>The emergence of artificial intelligence (AI) has allowed users to have access to large sources of information in a chat-like manner. Thereby, we sought to evaluate ChatGPT-4 response’s accuracy to the 10 patient most frequently asked questions (FAQs) regarding anterior cruciate ligament (ACL) surgery.</p></div><div><h3>Methods</h3><p>A list of the top 10 FAQs pertaining to ACL surgery was created after conducting a search through all Sports Medicine Fellowship Institutions listed on the Arthroscopy Association of North America (AANA) and American Orthopaedic Society of Sports Medicine (AOSSM) websites. A Likert scale was used to grade response accuracy by two sports medicine fellowship-trained surgeons. Cohen’s kappa was used to assess inter-rater agreement. Reproducibility of the responses over time was also assessed.</p></div><div><h3>Results</h3><p>Five of the 10 responses received a ‘completely accurate’ grade by two-fellowship trained surgeons with three additional replies receiving a ‘completely accurate’ status by at least one. Moreover, inter-rater reliability accuracy assessment revealed a moderate agreement between fellowship-trained attending physicians (weighted kappa = 0.57, 95% confidence interval 0.15–0.99). Additionally, 80% of the responses were reproducible over time.</p></div><div><h3>Conclusion</h3><p>ChatGPT can be considered an accurate additional tool to answer general patient questions regarding ACL surgery. None the less, patient–surgeon interaction should not be deferred and must continue to be the driving force for information retrieval. Thus, the general recommendation is to address any questions in the presence of a qualified specialist.</p></div>\",\"PeriodicalId\":56110,\"journal\":{\"name\":\"Knee\",\"volume\":\"51 \",\"pages\":\"Pages 84-92\"},\"PeriodicalIF\":1.6000,\"publicationDate\":\"2024-09-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Knee\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0968016024001480\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"ORTHOPEDICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knee","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0968016024001480","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ORTHOPEDICS","Score":null,"Total":0}
Accuracy assessment of ChatGPT responses to frequently asked questions regarding anterior cruciate ligament surgery
Background
The emergence of artificial intelligence (AI) has allowed users to have access to large sources of information in a chat-like manner. Thereby, we sought to evaluate ChatGPT-4 response’s accuracy to the 10 patient most frequently asked questions (FAQs) regarding anterior cruciate ligament (ACL) surgery.
Methods
A list of the top 10 FAQs pertaining to ACL surgery was created after conducting a search through all Sports Medicine Fellowship Institutions listed on the Arthroscopy Association of North America (AANA) and American Orthopaedic Society of Sports Medicine (AOSSM) websites. A Likert scale was used to grade response accuracy by two sports medicine fellowship-trained surgeons. Cohen’s kappa was used to assess inter-rater agreement. Reproducibility of the responses over time was also assessed.
Results
Five of the 10 responses received a ‘completely accurate’ grade by two-fellowship trained surgeons with three additional replies receiving a ‘completely accurate’ status by at least one. Moreover, inter-rater reliability accuracy assessment revealed a moderate agreement between fellowship-trained attending physicians (weighted kappa = 0.57, 95% confidence interval 0.15–0.99). Additionally, 80% of the responses were reproducible over time.
Conclusion
ChatGPT can be considered an accurate additional tool to answer general patient questions regarding ACL surgery. None the less, patient–surgeon interaction should not be deferred and must continue to be the driving force for information retrieval. Thus, the general recommendation is to address any questions in the presence of a qualified specialist.
期刊介绍：
The Knee is an international journal publishing studies on the clinical treatment and fundamental biomechanical characteristics of this joint. The aim of the journal is to provide a vehicle relevant to surgeons, biomedical engineers, imaging specialists, materials scientists, rehabilitation personnel and all those with an interest in the knee.
The topics covered include, but are not limited to:
• Anatomy, physiology, morphology and biochemistry;
• Biomechanical studies;
• Advances in the development of prosthetic, orthotic and augmentation devices;
• Imaging and diagnostic techniques;
• Pathology;
• Trauma;
• Surgery;
• Rehabilitation.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
