Appropriateness and Comprehensiveness of Using ChatGPT for Perioperative Patient Education in Thoracic Surgery in Different Language Contexts: Survey Study.
{"title":"Appropriateness and Comprehensiveness of Using ChatGPT for Perioperative Patient Education in Thoracic Surgery in Different Language Contexts: Survey Study.","authors":"Chen-Ye Shao, Hui Li, Xiao-Long Liu, Chang Li, Li-Qin Yang, Yue-Juan Zhang, Jing Luo, Jun Zhao","doi":"10.2196/46900","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>ChatGPT, a dialogue-based artificial intelligence language model, has shown promise in assisting clinical workflows and patient-clinician communication. However, there is a lack of feasibility assessments regarding its use for perioperative patient education in thoracic surgery.</p><p><strong>Objective: </strong>This study aimed to assess the appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts.</p><p><strong>Methods: </strong>This pilot study was conducted in February 2023. A total of 37 questions focused on perioperative patient education in thoracic surgery were created based on guidelines and clinical experience. Two sets of inquiries were made to ChatGPT for each question, one in English and the other in Chinese. The responses generated by ChatGPT were evaluated separately by experienced thoracic surgical clinicians for appropriateness and comprehensiveness based on a hypothetical draft response to a patient's question on the electronic information platform. For a response to be qualified, it required at least 80% of reviewers to deem it appropriate and 50% to deem it comprehensive. Statistical analyses were performed using the unpaired chi-square test or Fisher exact test, with a significance level set at P<.05.</p><p><strong>Results: </strong>The set of 37 commonly asked questions covered topics such as disease information, diagnostic procedures, perioperative complications, treatment measures, disease prevention, and perioperative care considerations. In both the English and Chinese contexts, 34 (92%) out of 37 responses were qualified in terms of both appropriateness and comprehensiveness. The remaining 3 (8%) responses were unqualified in these 2 contexts. The unqualified responses primarily involved the diagnosis of disease symptoms and surgical-related complications symptoms. The reasons for determining the responses as unqualified were similar in both contexts. There was no statistically significant difference (34/37, 92% vs 34/37, 92%; P=.99) in the qualification rate between the 2 language sets.</p><p><strong>Conclusions: </strong>This pilot study demonstrates the potential feasibility of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts. ChatGPT is expected to enhance patient satisfaction, reduce anxiety, and improve compliance during the perioperative period. In the future, there will be remarkable potential application for using artificial intelligence, in conjunction with human review, for patient education and health consultation after patients have provided their informed consent.</p>","PeriodicalId":51757,"journal":{"name":"Interactive Journal of Medical Research","volume":"12 ","pages":"e46900"},"PeriodicalIF":1.9000,"publicationDate":"2023-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10463083/pdf/","citationCount":"3","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Interactive Journal of Medical Research","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2196/46900","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MEDICINE, RESEARCH & EXPERIMENTAL","Score":null,"Total":0}
引用次数: 3
Abstract
Background: ChatGPT, a dialogue-based artificial intelligence language model, has shown promise in assisting clinical workflows and patient-clinician communication. However, there is a lack of feasibility assessments regarding its use for perioperative patient education in thoracic surgery.
Objective: This study aimed to assess the appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts.
Methods: This pilot study was conducted in February 2023. A total of 37 questions focused on perioperative patient education in thoracic surgery were created based on guidelines and clinical experience. Two sets of inquiries were made to ChatGPT for each question, one in English and the other in Chinese. The responses generated by ChatGPT were evaluated separately by experienced thoracic surgical clinicians for appropriateness and comprehensiveness based on a hypothetical draft response to a patient's question on the electronic information platform. For a response to be qualified, it required at least 80% of reviewers to deem it appropriate and 50% to deem it comprehensive. Statistical analyses were performed using the unpaired chi-square test or Fisher exact test, with a significance level set at P<.05.
Results: The set of 37 commonly asked questions covered topics such as disease information, diagnostic procedures, perioperative complications, treatment measures, disease prevention, and perioperative care considerations. In both the English and Chinese contexts, 34 (92%) out of 37 responses were qualified in terms of both appropriateness and comprehensiveness. The remaining 3 (8%) responses were unqualified in these 2 contexts. The unqualified responses primarily involved the diagnosis of disease symptoms and surgical-related complications symptoms. The reasons for determining the responses as unqualified were similar in both contexts. There was no statistically significant difference (34/37, 92% vs 34/37, 92%; P=.99) in the qualification rate between the 2 language sets.
Conclusions: This pilot study demonstrates the potential feasibility of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts. ChatGPT is expected to enhance patient satisfaction, reduce anxiety, and improve compliance during the perioperative period. In the future, there will be remarkable potential application for using artificial intelligence, in conjunction with human review, for patient education and health consultation after patients have provided their informed consent.
背景:ChatGPT是一种基于对话的人工智能语言模型,在辅助临床工作流程和患者-临床医生沟通方面显示出前景。然而,缺乏对胸外科围手术期患者教育的可行性评估。目的:本研究旨在评估中英文胸外科围手术期患者教育中使用ChatGPT的适宜性和全面性。方法:本研究于2023年2月进行。根据指南和临床经验,编制了37个胸外科围手术期患者教育问题。每个问题都向ChatGPT进行了两组查询,一组用英文,另一组用中文。ChatGPT生成的回复由经验丰富的胸外科医生根据对电子信息平台上患者问题的假设回复草稿,分别评估其适当性和全面性。对于合格的回应,至少需要80%的审稿人认为它是适当的,50%的审稿人认为它是全面的。采用非配对卡方检验或Fisher精确检验进行统计分析,显著性水平为:37个常见问题涵盖疾病信息、诊断程序、围手术期并发症、治疗措施、疾病预防和围手术期护理注意事项等主题。在中英文语境中,37个回答中有34个(92%)在适当性和全面性方面都是合格的。其余3个(8%)回答在这两种情况下是不合格的。不合格反应主要涉及疾病症状和手术相关并发症症状的诊断。在这两种情况下,将答复确定为不合格的原因是相似的。差异无统计学意义(34/ 37,92% vs 34/ 37,92%;P=.99)。结论:本初步研究证明了在中英文背景下使用ChatGPT进行胸外科围手术期患者教育的潜在可行性。ChatGPT有望在围手术期提高患者满意度,减少焦虑,提高依从性。在未来,将人工智能与人类审查相结合,在患者提供知情同意后进行患者教育和健康咨询,将有显著的潜在应用。