首页 > 最新文献

JMIR Medical Education最新文献

英文 中文
Leveraging Open-Source Large Language Models for Data Augmentation in Hospital Staff Surveys: Mixed Methods Study. 在医院员工调查中利用开源大型语言模型进行数据扩充:混合方法研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-19 DOI: 10.2196/51433
Carl Ehrett, Sudeep Hegde, Kwame Andre, Dixizi Liu, Timothy Wilson

Background: Generative large language models (LLMs) have the potential to revolutionize medical education by generating tailored learning materials, enhancing teaching efficiency, and improving learner engagement. However, the application of LLMs in health care settings, particularly for augmenting small datasets in text classification tasks, remains underexplored, particularly for cost- and privacy-conscious applications that do not permit the use of third-party services such as OpenAI's ChatGPT.

Objective: This study aims to explore the use of open-source LLMs, such as Large Language Model Meta AI (LLaMA) and Alpaca models, for data augmentation in a specific text classification task related to hospital staff surveys.

Methods: The surveys were designed to elicit narratives of everyday adaptation by frontline radiology staff during the initial phase of the COVID-19 pandemic. A 2-step process of data augmentation and text classification was conducted. The study generated synthetic data similar to the survey reports using 4 generative LLMs for data augmentation. A different set of 3 classifier LLMs was then used to classify the augmented text for thematic categories. The study evaluated performance on the classification task.

Results: The overall best-performing combination of LLMs, temperature, classifier, and number of synthetic data cases is via augmentation with LLaMA 7B at temperature 0.7 with 100 augments, using Robustly Optimized BERT Pretraining Approach (RoBERTa) for the classification task, achieving an average area under the receiver operating characteristic (AUC) curve of 0.87 (SD 0.02; ie, 1 SD). The results demonstrate that open-source LLMs can enhance text classifiers' performance for small datasets in health care contexts, providing promising pathways for improving medical education processes and patient care practices.

Conclusions: The study demonstrates the value of data augmentation with open-source LLMs, highlights the importance of privacy and ethical considerations when using LLMs, and suggests future directions for research in this field.

背景:生成式大型语言模型(LLMs)可生成量身定制的学习材料、提高教学效率并改善学习者的参与度,从而有望彻底改变医学教育。然而,LLMs 在医疗环境中的应用,尤其是在文本分类任务中用于扩充小型数据集的应用,仍未得到充分探索,特别是在成本和隐私意识较高的应用中,因为这些应用不允许使用第三方服务,如 OpenAI 的 ChatGPT:本研究旨在探索开源 LLM(如大型语言模型元人工智能(LLaMA)和 Alpaca 模型)在与医院员工调查相关的特定文本分类任务中的数据增强应用:调查旨在了解一线放射科工作人员在 COVID-19 大流行初期的日常适应情况。研究采用了数据扩充和文本分类两个步骤。研究使用 4 个生成式 LLM 生成与调查报告类似的合成数据,用于数据扩增。然后使用一组不同的 3 个分类器 LLM 对增强文本进行主题分类。研究评估了分类任务的性能:LLMs、温度、分类器和合成数据个数的最佳组合是在温度为 0.7 的条件下使用 LLaMA 7B 进行扩增,并使用稳健优化的 BERT 预训练方法 (RoBERTa) 进行 100 个扩增,从而完成分类任务,其接收者操作特征曲线下的平均面积 (AUC) 为 0.87(SD 0.02;即 1 SD)。研究结果表明,开源 LLM 可以提高医疗保健领域小型数据集文本分类器的性能,为改善医学教育流程和患者护理实践提供了前景广阔的途径:本研究证明了使用开源 LLMs 增强数据的价值,强调了使用 LLMs 时隐私和伦理考虑的重要性,并提出了该领域未来的研究方向。
{"title":"Leveraging Open-Source Large Language Models for Data Augmentation in Hospital Staff Surveys: Mixed Methods Study.","authors":"Carl Ehrett, Sudeep Hegde, Kwame Andre, Dixizi Liu, Timothy Wilson","doi":"10.2196/51433","DOIUrl":"10.2196/51433","url":null,"abstract":"<p><strong>Background: </strong>Generative large language models (LLMs) have the potential to revolutionize medical education by generating tailored learning materials, enhancing teaching efficiency, and improving learner engagement. However, the application of LLMs in health care settings, particularly for augmenting small datasets in text classification tasks, remains underexplored, particularly for cost- and privacy-conscious applications that do not permit the use of third-party services such as OpenAI's ChatGPT.</p><p><strong>Objective: </strong>This study aims to explore the use of open-source LLMs, such as Large Language Model Meta AI (LLaMA) and Alpaca models, for data augmentation in a specific text classification task related to hospital staff surveys.</p><p><strong>Methods: </strong>The surveys were designed to elicit narratives of everyday adaptation by frontline radiology staff during the initial phase of the COVID-19 pandemic. A 2-step process of data augmentation and text classification was conducted. The study generated synthetic data similar to the survey reports using 4 generative LLMs for data augmentation. A different set of 3 classifier LLMs was then used to classify the augmented text for thematic categories. The study evaluated performance on the classification task.</p><p><strong>Results: </strong>The overall best-performing combination of LLMs, temperature, classifier, and number of synthetic data cases is via augmentation with LLaMA 7B at temperature 0.7 with 100 augments, using Robustly Optimized BERT Pretraining Approach (RoBERTa) for the classification task, achieving an average area under the receiver operating characteristic (AUC) curve of 0.87 (SD 0.02; ie, 1 SD). The results demonstrate that open-source LLMs can enhance text classifiers' performance for small datasets in health care contexts, providing promising pathways for improving medical education processes and patient care practices.</p><p><strong>Conclusions: </strong>The study demonstrates the value of data augmentation with open-source LLMs, highlights the importance of privacy and ethical considerations when using LLMs, and suggests future directions for research in this field.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e51433"},"PeriodicalIF":3.2,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142669265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Virtual Reality Simulation in Undergraduate Health Care Education Programs: Usability Study. 本科医疗保健教育课程中的虚拟现实模拟:可用性研究
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-19 DOI: 10.2196/56844
Gry Mørk, Tore Bonsaksen, Ole Sønnik Larsen, Hans Martin Kunnikoff, Silje Stangeland Lie

Background: Virtual reality (VR) is increasingly being used in higher education for clinical skills training and role-playing among health care students. Using 360° videos in VR headsets, followed by peer debrief and group discussions, may strengthen students' social and emotional learning.

Objective: This study aimed to explore student-perceived usability of VR simulation in three health care education programs in Norway.

Methods: Students from one university participated in a VR simulation program. Of these, students in social education (n=74), nursing (n=45), and occupational therapy (n=27) completed a questionnaire asking about their perceptions of the usability of the VR simulation and the related learning activities. Differences between groups of students were examined with Pearson chi-square tests and with 1-way ANOVA. Qualitative content analysis was used to analyze data from open-ended questions.

Results: The nursing students were most satisfied with the usability of the VR simulation, while the occupational therapy students were least satisfied. The nursing students had more often prior experience from using VR technology (60%), while occupational therapy students less often had prior experience (37%). Nevertheless, high mean scores indicated that the students experienced the VR simulation and the related learning activities as very useful. The results also showed that by using realistic scenarios in VR simulation, health care students can be prepared for complex clinical situations in a safe environment. Also, group debriefing sessions are a vital part of the learning process that enhance active involvement with peers.

Conclusions: VR simulation has promise and potential as a pedagogical tool in health care education, especially for training soft skills relevant for clinical practice, such as communication, decision-making, time management, and critical thinking.

背景:虚拟现实(VR)越来越多地应用于高等教育中的临床技能培训和医护学生的角色扮演。在 VR 头显中使用 360° 视频,然后进行同伴汇报和小组讨论,可以加强学生的社会和情感学习:本研究旨在探讨挪威三个医疗保健教育项目中学生对 VR 模拟可用性的看法:方法:一所大学的学生参加了VR模拟项目。其中,社会教育专业(74人)、护理专业(45人)和职业治疗专业(27人)的学生填写了一份调查问卷,询问他们对VR模拟和相关学习活动可用性的看法。采用皮尔逊卡方检验和单因素方差分析来检验学生组间的差异。定性内容分析法用于分析开放式问题中的数据:结果:护理专业学生对 VR 模拟的可用性最为满意,而职业治疗专业学生的满意度最低。护理专业的学生更经常使用 VR 技术(60%),而职业治疗专业的学生则较少使用(37%)。尽管如此,高平均分表明学生们认为 VR 模拟和相关学习活动非常有用。结果还显示,通过在 VR 模拟中使用逼真的场景,医护学生可以在安全的环境中为复杂的临床情况做好准备。此外,小组汇报环节也是学习过程中的重要组成部分,能增强学生与同伴的积极参与:VR模拟作为医疗保健教育的一种教学工具,特别是在培训与临床实践相关的软技能(如沟通、决策、时间管理和批判性思维)方面,具有广阔的前景和潜力。
{"title":"Virtual Reality Simulation in Undergraduate Health Care Education Programs: Usability Study.","authors":"Gry Mørk, Tore Bonsaksen, Ole Sønnik Larsen, Hans Martin Kunnikoff, Silje Stangeland Lie","doi":"10.2196/56844","DOIUrl":"10.2196/56844","url":null,"abstract":"<p><strong>Background: </strong>Virtual reality (VR) is increasingly being used in higher education for clinical skills training and role-playing among health care students. Using 360° videos in VR headsets, followed by peer debrief and group discussions, may strengthen students' social and emotional learning.</p><p><strong>Objective: </strong>This study aimed to explore student-perceived usability of VR simulation in three health care education programs in Norway.</p><p><strong>Methods: </strong>Students from one university participated in a VR simulation program. Of these, students in social education (n=74), nursing (n=45), and occupational therapy (n=27) completed a questionnaire asking about their perceptions of the usability of the VR simulation and the related learning activities. Differences between groups of students were examined with Pearson chi-square tests and with 1-way ANOVA. Qualitative content analysis was used to analyze data from open-ended questions.</p><p><strong>Results: </strong>The nursing students were most satisfied with the usability of the VR simulation, while the occupational therapy students were least satisfied. The nursing students had more often prior experience from using VR technology (60%), while occupational therapy students less often had prior experience (37%). Nevertheless, high mean scores indicated that the students experienced the VR simulation and the related learning activities as very useful. The results also showed that by using realistic scenarios in VR simulation, health care students can be prepared for complex clinical situations in a safe environment. Also, group debriefing sessions are a vital part of the learning process that enhance active involvement with peers.</p><p><strong>Conclusions: </strong>VR simulation has promise and potential as a pedagogical tool in health care education, especially for training soft skills relevant for clinical practice, such as communication, decision-making, time management, and critical thinking.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e56844"},"PeriodicalIF":3.2,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142669267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Correction: Psychological Safety Competency Training During the Clinical Internship From the Perspective of Health Care Trainee Mentors in 11 Pan-European Countries: Mixed Methods Observational Study. 更正:从 11 个泛欧国家医疗保健实习生导师的角度看临床实习期间的心理安全能力培训:混合方法观察研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-15 DOI: 10.2196/68503
Irene Carrillo, Ivana Skoumalová, Ireen Bruus, Victoria Klemm, Sofia Guerra-Paiva, Bojana Knežević, Augustina Jankauskiene, Dragana Jocic, Susanna Tella, Sandra C Buttigieg, Einav Srulovici, Andrea Madarasová Gecková, Kaja Põlluste, Reinhard Strametz, Paulo Sousa, Marina Odalovic, José Joaquín Mira

[This corrects the article DOI: 10.2196/64125.].

[此处更正了文章 DOI:10.2196/64125]。
{"title":"Correction: Psychological Safety Competency Training During the Clinical Internship From the Perspective of Health Care Trainee Mentors in 11 Pan-European Countries: Mixed Methods Observational Study.","authors":"Irene Carrillo, Ivana Skoumalová, Ireen Bruus, Victoria Klemm, Sofia Guerra-Paiva, Bojana Knežević, Augustina Jankauskiene, Dragana Jocic, Susanna Tella, Sandra C Buttigieg, Einav Srulovici, Andrea Madarasová Gecková, Kaja Põlluste, Reinhard Strametz, Paulo Sousa, Marina Odalovic, José Joaquín Mira","doi":"10.2196/68503","DOIUrl":"10.2196/68503","url":null,"abstract":"<p><p>[This corrects the article DOI: 10.2196/64125.].</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e68503"},"PeriodicalIF":3.2,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142639999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Leveraging the Electronic Health Record to Measure Resident Clinical Experiences and Identify Training Gaps: Development and Usability Study. 利用电子健康记录测量住院医师的临床经验并找出培训差距:开发和可用性研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-06 DOI: 10.2196/53337
Vasudha L Bhavaraju, Sarada Panchanathan, Brigham C Willis, Pamela Garcia-Filion

Background: Competence-based medical education requires robust data to link competence with clinical experiences. The SARS-CoV-2 (COVID-19) pandemic abruptly altered the standard trajectory of clinical exposure in medical training programs. Residency program directors were tasked with identifying and addressing the resultant gaps in each trainee's experiences using existing tools.

Objective: This study aims to demonstrate a feasible and efficient method to capture electronic health record (EHR) data that measure the volume and variety of pediatric resident clinical experiences from a continuity clinic; generate individual-, class-, and graduate-level benchmark data; and create a visualization for learners to quickly identify gaps in clinical experiences.

Methods: This pilot was conducted in a large, urban pediatric residency program from 2016 to 2022. Through consensus, 5 pediatric faculty identified diagnostic groups that pediatric residents should see to be competent in outpatient pediatrics. Information technology consultants used International Classification of Diseases, Tenth Revision (ICD-10) codes corresponding with each diagnostic group to extract EHR patient encounter data as an indicator of exposure to the specific diagnosis. The frequency (volume) and diagnosis types (variety) seen by active residents (classes of 2020-2022) were compared with class and graduated resident (classes of 2016-2019) averages. These data were converted to percentages and translated to a radar chart visualization for residents to quickly compare their current clinical experiences with peers and graduates. Residents were surveyed on the use of these data and the visualization to identify training gaps.

Results: Patient encounter data about clinical experiences for 102 residents (N=52 graduates) were extracted. Active residents (n=50) received data reports with radar graphs biannually: 3 for the classes of 2020 and 2021 and 2 for the class of 2022. Radar charts distinctly demonstrated gaps in diagnoses exposure compared with classmates and graduates. Residents found the visualization useful in setting clinical and learning goals.

Conclusions: This pilot describes an innovative method of capturing and presenting data about resident clinical experiences, compared with peer and graduate benchmarks, to identify learning gaps that may result from disruptions or modifications in medical training. This methodology can be aggregated across specialties and institutions and potentially inform competence-based medical education.

背景:以能力为基础的医学教育需要可靠的数据将能力与临床经验联系起来。SARS-CoV-2(COVID-19)大流行突然改变了医学培训项目中临床接触的标准轨迹。住院医师培训项目主任的任务是利用现有工具找出并解决每个学员的经验差距:本研究旨在展示一种可行且高效的方法,用于采集电子健康记录(EHR)数据,以衡量连续性诊所儿科住院医师临床经验的数量和多样性;生成个人、班级和研究生水平的基准数据;并为学员创建可视化工具,以快速识别临床经验中的差距:该试点项目于 2016 年至 2022 年在一个大型城市儿科住院医师培训项目中开展。通过达成共识,5 位儿科教师确定了儿科住院医师为胜任儿科门诊工作而应看的诊断组别。信息技术顾问使用与每个诊断组相对应的《国际疾病分类第十版》(ICD-10)代码提取电子病历患者就诊数据,作为接触特定诊断的指标。在职住院医师(2020-2022 届)的就诊频率(数量)和诊断类型(种类)与班级和毕业住院医师(2016-2019 届)的平均值进行了比较。这些数据被转换为百分比,并转化为雷达图可视化,以便住院医师将其当前的临床经验与同级住院医师和毕业生进行快速比较。住院医师接受了关于使用这些数据和可视化图表找出培训差距的调查:提取了 102 名住院医师(毕业生人数=52)的临床经验数据。在职住院医师(人数=50)每半年收到一次带有雷达图的数据报告:2020届和2021届各3份,2022届2份。雷达图明显显示了与同学和毕业生相比在诊断暴露方面的差距。住院医师发现,这种可视化方法有助于制定临床和学习目标:本试验介绍了一种创新方法,通过与同学和毕业生的基准进行比较,获取并展示住院医师临床经验的数据,从而找出因医学培训中断或修改而可能导致的学习差距。这种方法可以在各专科和机构间进行汇总,并有可能为基于能力的医学教育提供参考。
{"title":"Leveraging the Electronic Health Record to Measure Resident Clinical Experiences and Identify Training Gaps: Development and Usability Study.","authors":"Vasudha L Bhavaraju, Sarada Panchanathan, Brigham C Willis, Pamela Garcia-Filion","doi":"10.2196/53337","DOIUrl":"10.2196/53337","url":null,"abstract":"<p><strong>Background: </strong>Competence-based medical education requires robust data to link competence with clinical experiences. The SARS-CoV-2 (COVID-19) pandemic abruptly altered the standard trajectory of clinical exposure in medical training programs. Residency program directors were tasked with identifying and addressing the resultant gaps in each trainee's experiences using existing tools.</p><p><strong>Objective: </strong>This study aims to demonstrate a feasible and efficient method to capture electronic health record (EHR) data that measure the volume and variety of pediatric resident clinical experiences from a continuity clinic; generate individual-, class-, and graduate-level benchmark data; and create a visualization for learners to quickly identify gaps in clinical experiences.</p><p><strong>Methods: </strong>This pilot was conducted in a large, urban pediatric residency program from 2016 to 2022. Through consensus, 5 pediatric faculty identified diagnostic groups that pediatric residents should see to be competent in outpatient pediatrics. Information technology consultants used International Classification of Diseases, Tenth Revision (ICD-10) codes corresponding with each diagnostic group to extract EHR patient encounter data as an indicator of exposure to the specific diagnosis. The frequency (volume) and diagnosis types (variety) seen by active residents (classes of 2020-2022) were compared with class and graduated resident (classes of 2016-2019) averages. These data were converted to percentages and translated to a radar chart visualization for residents to quickly compare their current clinical experiences with peers and graduates. Residents were surveyed on the use of these data and the visualization to identify training gaps.</p><p><strong>Results: </strong>Patient encounter data about clinical experiences for 102 residents (N=52 graduates) were extracted. Active residents (n=50) received data reports with radar graphs biannually: 3 for the classes of 2020 and 2021 and 2 for the class of 2022. Radar charts distinctly demonstrated gaps in diagnoses exposure compared with classmates and graduates. Residents found the visualization useful in setting clinical and learning goals.</p><p><strong>Conclusions: </strong>This pilot describes an innovative method of capturing and presenting data about resident clinical experiences, compared with peer and graduate benchmarks, to identify learning gaps that may result from disruptions or modifications in medical training. This methodology can be aggregated across specialties and institutions and potentially inform competence-based medical education.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e53337"},"PeriodicalIF":3.2,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11559912/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142591681","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
ChatGPT-4 Omni Performance in USMLE Disciplines and Clinical Skills: Comparative Analysis. ChatGPT-4 Omni 在 USMLE 学科和临床技能中的表现:比较分析。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-06 DOI: 10.2196/63430
Brenton T Bicknell, Danner Butler, Sydney Whalen, James Ricks, Cory J Dixon, Abigail B Clark, Olivia Spaedy, Adam Skelton, Neel Edupuganti, Lance Dzubinski, Hudson Tate, Garrett Dyess, Brenessa Lindeman, Lisa Soleymani Lehmann

Background: Recent studies, including those by the National Board of Medical Examiners, have highlighted the remarkable capabilities of recent large language models (LLMs) such as ChatGPT in passing the United States Medical Licensing Examination (USMLE). However, there is a gap in detailed analysis of LLM performance in specific medical content areas, thus limiting an assessment of their potential utility in medical education.

Objective: This study aimed to assess and compare the accuracy of successive ChatGPT versions (GPT-3.5, GPT-4, and GPT-4 Omni) in USMLE disciplines, clinical clerkships, and the clinical skills of diagnostics and management.

Methods: This study used 750 clinical vignette-based multiple-choice questions to characterize the performance of successive ChatGPT versions (ChatGPT 3.5 [GPT-3.5], ChatGPT 4 [GPT-4], and ChatGPT 4 Omni [GPT-4o]) across USMLE disciplines, clinical clerkships, and in clinical skills (diagnostics and management). Accuracy was assessed using a standardized protocol, with statistical analyses conducted to compare the models' performances.

Results: GPT-4o achieved the highest accuracy across 750 multiple-choice questions at 90.4%, outperforming GPT-4 and GPT-3.5, which scored 81.1% and 60.0%, respectively. GPT-4o's highest performances were in social sciences (95.5%), behavioral and neuroscience (94.2%), and pharmacology (93.2%). In clinical skills, GPT-4o's diagnostic accuracy was 92.7% and management accuracy was 88.8%, significantly higher than its predecessors. Notably, both GPT-4o and GPT-4 significantly outperformed the medical student average accuracy of 59.3% (95% CI 58.3-60.3).

Conclusions: GPT-4o's performance in USMLE disciplines, clinical clerkships, and clinical skills indicates substantial improvements over its predecessors, suggesting significant potential for the use of this technology as an educational aid for medical students. These findings underscore the need for careful consideration when integrating LLMs into medical education, emphasizing the importance of structured curricula to guide their appropriate use and the need for ongoing critical analyses to ensure their reliability and effectiveness.

背景:最近的研究,包括美国国家医学考试委员会(National Board of Medical Examiners)的研究,都强调了最近的大型语言模型(LLM),如 ChatGPT,在通过美国医学执照考试(USMLE)方面的卓越能力。然而,在详细分析 LLM 在特定医学内容领域的表现方面还存在空白,从而限制了对其在医学教育中潜在用途的评估:本研究旨在评估和比较历代 ChatGPT 版本(GPT-3.5、GPT-4 和 GPT-4 Omni)在 USMLE 学科、临床实习以及诊断和管理临床技能方面的准确性:本研究使用了 750 道基于临床小故事的选择题,以描述历代 ChatGPT 版本(ChatGPT 3.5 [GPT-3.5]、ChatGPT 4 [GPT-4]和 ChatGPT 4 Omni [GPT-4o])在 USMLE 学科、临床实习和临床技能(诊断和管理)方面的表现。采用标准化方案评估准确性,并进行统计分析以比较模型的性能:结果:在750道选择题中,GPT-4o的准确率最高,达到90.4%,超过了分别为81.1%和60.0%的GPT-4和GPT-3.5。GPT-4o 在社会科学(95.5%)、行为与神经科学(94.2%)和药理学(93.2%)方面表现最佳。在临床技能方面,GPT-4o 的诊断准确率为 92.7%,管理准确率为 88.8%,明显高于其前身。值得注意的是,GPT-4o和GPT-4的准确率均明显高于医学生59.3%(95% CI 58.3-60.3)的平均准确率:结论:GPT-4o 在 USMLE 学科、临床实习和临床技能方面的表现比其前代产品有了大幅提高,这表明该技术作为医学生教育辅助工具的巨大潜力。这些发现强调了在将 LLMs 纳入医学教育时需要慎重考虑的问题,强调了结构化课程对指导其合理使用的重要性,以及持续进行关键分析以确保其可靠性和有效性的必要性。
{"title":"ChatGPT-4 Omni Performance in USMLE Disciplines and Clinical Skills: Comparative Analysis.","authors":"Brenton T Bicknell, Danner Butler, Sydney Whalen, James Ricks, Cory J Dixon, Abigail B Clark, Olivia Spaedy, Adam Skelton, Neel Edupuganti, Lance Dzubinski, Hudson Tate, Garrett Dyess, Brenessa Lindeman, Lisa Soleymani Lehmann","doi":"10.2196/63430","DOIUrl":"https://doi.org/10.2196/63430","url":null,"abstract":"<p><strong>Background: </strong>Recent studies, including those by the National Board of Medical Examiners, have highlighted the remarkable capabilities of recent large language models (LLMs) such as ChatGPT in passing the United States Medical Licensing Examination (USMLE). However, there is a gap in detailed analysis of LLM performance in specific medical content areas, thus limiting an assessment of their potential utility in medical education.</p><p><strong>Objective: </strong>This study aimed to assess and compare the accuracy of successive ChatGPT versions (GPT-3.5, GPT-4, and GPT-4 Omni) in USMLE disciplines, clinical clerkships, and the clinical skills of diagnostics and management.</p><p><strong>Methods: </strong>This study used 750 clinical vignette-based multiple-choice questions to characterize the performance of successive ChatGPT versions (ChatGPT 3.5 [GPT-3.5], ChatGPT 4 [GPT-4], and ChatGPT 4 Omni [GPT-4o]) across USMLE disciplines, clinical clerkships, and in clinical skills (diagnostics and management). Accuracy was assessed using a standardized protocol, with statistical analyses conducted to compare the models' performances.</p><p><strong>Results: </strong>GPT-4o achieved the highest accuracy across 750 multiple-choice questions at 90.4%, outperforming GPT-4 and GPT-3.5, which scored 81.1% and 60.0%, respectively. GPT-4o's highest performances were in social sciences (95.5%), behavioral and neuroscience (94.2%), and pharmacology (93.2%). In clinical skills, GPT-4o's diagnostic accuracy was 92.7% and management accuracy was 88.8%, significantly higher than its predecessors. Notably, both GPT-4o and GPT-4 significantly outperformed the medical student average accuracy of 59.3% (95% CI 58.3-60.3).</p><p><strong>Conclusions: </strong>GPT-4o's performance in USMLE disciplines, clinical clerkships, and clinical skills indicates substantial improvements over its predecessors, suggesting significant potential for the use of this technology as an educational aid for medical students. These findings underscore the need for careful consideration when integrating LLMs into medical education, emphasizing the importance of structured curricula to guide their appropriate use and the need for ongoing critical analyses to ensure their reliability and effectiveness.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e63430"},"PeriodicalIF":3.2,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142591666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Potential of Artificial Intelligence Tools for Reducing Uncertainty in Medicine and Directions for Medical Education. 人工智能工具在减少医学不确定性方面的潜力及医学教育的方向》(The Potential of Artificial Intelligence Tools for Reducing Ununcertainty in Medicine and Directions for Medical Education)。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-11-04 DOI: 10.2196/51446
Sauliha Rabia Alli, Soaad Qahhār Hossain, Sunit Das, Ross Upshur

Unlabelled: In the field of medicine, uncertainty is inherent. Physicians are asked to make decisions on a daily basis without complete certainty, whether it is in understanding the patient's problem, performing the physical examination, interpreting the findings of diagnostic tests, or proposing a management plan. The reasons for this uncertainty are widespread, including the lack of knowledge about the patient, individual physician limitations, and the limited predictive power of objective diagnostic tools. This uncertainty poses significant problems in providing competent patient care. Research efforts and teaching are attempts to reduce uncertainty that have now become inherent to medicine. Despite this, uncertainty is rampant. Artificial intelligence (AI) tools, which are being rapidly developed and integrated into practice, may change the way we navigate uncertainty. In their strongest forms, AI tools may have the ability to improve data collection on diseases, patient beliefs, values, and preferences, thereby allowing more time for physician-patient communication. By using methods not previously considered, these tools hold the potential to reduce the uncertainty in medicine, such as those arising due to the lack of clinical information and provider skill and bias. Despite this possibility, there has been considerable resistance to the implementation of AI tools in medical practice. In this viewpoint article, we discuss the impact of AI on medical uncertainty and discuss practical approaches to teaching the use of AI tools in medical schools and residency training programs, including AI ethics, real-world skills, and technological aptitude.

无标签:在医学领域,不确定性是与生俱来的。医生每天都要在没有十足把握的情况下做出决定,无论是在了解病人的问题、进行体格检查、解释诊断检测结果还是提出治疗方案方面。造成这种不确定性的原因很多,包括对病人缺乏了解、医生个人能力有限以及客观诊断工具的预测能力有限。这种不确定性给提供合格的病人护理带来了重大问题。研究工作和教学试图减少不确定性,这已成为医学的固有特点。尽管如此,不确定性依然猖獗。人工智能(AI)工具正在迅速发展并融入实践,它可能会改变我们驾驭不确定性的方式。在最强大的形式下,人工智能工具可能有能力改善有关疾病、患者信仰、价值观和偏好的数据收集,从而为医患沟通留出更多时间。通过使用以前未曾考虑过的方法,这些工具有可能减少医学中的不确定性,例如由于缺乏临床信息以及提供者的技能和偏见而产生的不确定性。尽管存在这种可能性,但在医疗实践中使用人工智能工具却遇到了相当大的阻力。在这篇观点文章中,我们讨论了人工智能对医学不确定性的影响,并探讨了在医学院和住院医师培训项目中教授使用人工智能工具的实用方法,包括人工智能伦理、实际技能和技术能力。
{"title":"The Potential of Artificial Intelligence Tools for Reducing Uncertainty in Medicine and Directions for Medical Education.","authors":"Sauliha Rabia Alli, Soaad Qahhār Hossain, Sunit Das, Ross Upshur","doi":"10.2196/51446","DOIUrl":"10.2196/51446","url":null,"abstract":"<p><strong>Unlabelled: </strong>In the field of medicine, uncertainty is inherent. Physicians are asked to make decisions on a daily basis without complete certainty, whether it is in understanding the patient's problem, performing the physical examination, interpreting the findings of diagnostic tests, or proposing a management plan. The reasons for this uncertainty are widespread, including the lack of knowledge about the patient, individual physician limitations, and the limited predictive power of objective diagnostic tools. This uncertainty poses significant problems in providing competent patient care. Research efforts and teaching are attempts to reduce uncertainty that have now become inherent to medicine. Despite this, uncertainty is rampant. Artificial intelligence (AI) tools, which are being rapidly developed and integrated into practice, may change the way we navigate uncertainty. In their strongest forms, AI tools may have the ability to improve data collection on diseases, patient beliefs, values, and preferences, thereby allowing more time for physician-patient communication. By using methods not previously considered, these tools hold the potential to reduce the uncertainty in medicine, such as those arising due to the lack of clinical information and provider skill and bias. Despite this possibility, there has been considerable resistance to the implementation of AI tools in medical practice. In this viewpoint article, we discuss the impact of AI on medical uncertainty and discuss practical approaches to teaching the use of AI tools in medical schools and residency training programs, including AI ethics, real-world skills, and technological aptitude.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e51446"},"PeriodicalIF":3.2,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11554287/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142575636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Transforming the Future of Digital Health Education: Redesign of a Graduate Program Using Competency Mapping. 改变数字健康教育的未来:利用能力图谱重新设计研究生课程。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-10-31 DOI: 10.2196/54112
Michelle Mun, Sonia Chanchlani, Kayley Lyons, Kathleen Gray

Unlabelled: Digital transformation has disrupted many industries but is yet to revolutionize health care. Educational programs must be aligned with the reality that goes beyond developing individuals in their own professions, professionals wishing to make an impact in digital health will need a multidisciplinary understanding of how business models, organizational processes, stakeholder relationships, and workforce dynamics across the health care ecosystem may be disrupted by digital health technology. This paper describes the redesign of an existing postgraduate program, ensuring that core digital health content is relevant, pedagogically sound, and evidence-based, and that the program provides learning and practical application of concepts of the digital transformation of health. Existing subjects were mapped to the American Medical Informatics Association Clinical Informatics Core Competencies, followed by consultation with leadership to further identify gaps or opportunities to revise the course structure. New additions of core and elective subjects were proposed to align with the competencies. Suitable electives were chosen based on stakeholder feedback and a review of subjects in fields relevant to digital transformation of health. The program was revised with a new title, course overview, course intended learning outcomes, reorganizing of core subjects, and approval of new electives, adding to a suite of professional development offerings and forming a structured pathway to further qualification. Programs in digital health must move beyond purely informatics-based competencies toward enabling transformational change. Postgraduate program development in this field is possible within a short time frame with the use of established competency frameworks and expert and student consultation.

无标签:数字化转型已经颠覆了许多行业,但尚未彻底改变医疗行业。教育计划必须与现实相一致,除了培养本专业的人才外,希望在数字医疗领域有所作为的专业人士还需要多学科的理解,即数字医疗技术可能会如何颠覆整个医疗生态系统中的商业模式、组织流程、利益相关者关系和劳动力动态。本文介绍了对现有研究生课程的重新设计,以确保数字医疗的核心内容具有相关性、教学合理性和循证性,并确保该课程能够提供医疗数字化转型概念的学习和实际应用。我们将现有科目与美国医学信息学协会临床信息学核心能力进行了映射,随后与领导层进行了磋商,以进一步确定差距或修改课程结构的机会。为了与这些能力保持一致,提出了新增核心科目和选修科目的建议。根据利益相关者的反馈以及对医疗数字化转型相关领域科目的审查,选择了合适的选修课。该计划经过修订,采用了新的名称、课程概述、课程预期学习成果,重新组织了核心科目,并批准了新的选修课,增加了一系列专业发展课程,形成了一条结构化的晋升途径。数字医疗课程必须超越纯粹的信息学能力,实现转型变革。利用已有的能力框架以及专家和学生咨询,可以在短时间内开发出该领域的研究生课程。
{"title":"Transforming the Future of Digital Health Education: Redesign of a Graduate Program Using Competency Mapping.","authors":"Michelle Mun, Sonia Chanchlani, Kayley Lyons, Kathleen Gray","doi":"10.2196/54112","DOIUrl":"10.2196/54112","url":null,"abstract":"<p><strong>Unlabelled: </strong>Digital transformation has disrupted many industries but is yet to revolutionize health care. Educational programs must be aligned with the reality that goes beyond developing individuals in their own professions, professionals wishing to make an impact in digital health will need a multidisciplinary understanding of how business models, organizational processes, stakeholder relationships, and workforce dynamics across the health care ecosystem may be disrupted by digital health technology. This paper describes the redesign of an existing postgraduate program, ensuring that core digital health content is relevant, pedagogically sound, and evidence-based, and that the program provides learning and practical application of concepts of the digital transformation of health. Existing subjects were mapped to the American Medical Informatics Association Clinical Informatics Core Competencies, followed by consultation with leadership to further identify gaps or opportunities to revise the course structure. New additions of core and elective subjects were proposed to align with the competencies. Suitable electives were chosen based on stakeholder feedback and a review of subjects in fields relevant to digital transformation of health. The program was revised with a new title, course overview, course intended learning outcomes, reorganizing of core subjects, and approval of new electives, adding to a suite of professional development offerings and forming a structured pathway to further qualification. Programs in digital health must move beyond purely informatics-based competencies toward enabling transformational change. Postgraduate program development in this field is possible within a short time frame with the use of established competency frameworks and expert and student consultation.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e54112"},"PeriodicalIF":3.2,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11542907/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142559031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Pilot Project to Promote Research Competency in Medical Students Through Journal Clubs: Mixed Methods Study. 通过期刊俱乐部提高医学生研究能力的试点项目:混合方法研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-10-31 DOI: 10.2196/51173
Mert Karabacak, Zeynep Ozcan, Burak Berksu Ozkara, Zeynep Sude Furkan, Sotirios Bisdas

Background: Undergraduate medical students often lack hands-on research experience and fundamental scientific research skills, limiting their exposure to the practical aspects of scientific investigation. The Cerrahpasa Neuroscience Society introduced a program to address this deficiency and facilitate student-led research.

Objective: The primary goal of this initiative was to enhance medical students' research output by enabling them to generate and publish peer-reviewed papers within the framework of this pilot project. The project aimed to provide an accessible, global model for research training through structured journal clubs, mentorship from experienced peers, and resource access.

Methods: In January 2022, a total of 30 volunteer students from various Turkish medical schools participated in this course-based undergraduate research experience program. Students self-organized into 2 groups according to their preferred study type: original research or systematic review. Two final-year students with prior research experience led the project, developing training modules using selected materials. The project was implemented entirely online, with participants completing training modules before using their newly acquired theoretical knowledge to perform assigned tasks.

Results: Based on student feedback, the project timeline was adjusted to allow for greater flexibility in meeting deadlines. Despite these adjustments, participants successfully completed their tasks, applying the theoretical knowledge they had gained to their respective assignments. As of April 2024, the initiative has culminated in 3 published papers and 3 more under peer review. The project has also seen an increase in student interest in further involvement and self-paced learning.

Conclusions: This initiative leverages globally accessible resources for research training, effectively fostering research competency among participants. It has successfully demonstrated the potential for undergraduates to contribute to medical research output and paved the way for a self-sustaining, student-led research program. Despite some logistical challenges, the project provided valuable insights for future implementations, showcasing the potential for students to engage in meaningful, publishable research.

背景:医科本科生往往缺乏实际研究经验和基本科学研究技能,这限制了他们接触科学研究的实践方面。Cerrahpasa 神经科学学会推出了一项计划,以解决这一不足,促进学生主导的研究:该计划的主要目标是提高医科学生的研究成果,使他们能够在试点项目框架内撰写并发表经同行评审的论文。该项目旨在通过有组织的期刊俱乐部、经验丰富的同行指导和资源获取,为研究培训提供一个可利用的全球模式:2022 年 1 月,共有 30 名来自土耳其不同医学院校的志愿学生参加了这个以课程为基础的本科生研究体验项目。学生们根据自己喜欢的研究类型自行分为两组:原创研究或系统综述。两名有研究经验的毕业班学生领导该项目,使用选定的材料开发培训模块。该项目完全在网上实施,参与者先完成培训模块,然后再利用新获得的理论知识完成指定任务:结果:根据学生的反馈意见,对项目时间表进行了调整,以便在截止日期前更灵活地完成任务。尽管进行了这些调整,学员们还是成功地完成了任务,将所学的理论知识应用到了各自的任务中。截至 2024 年 4 月,该项目已发表 3 篇论文,另有 3 篇论文正在接受同行评审。该项目还提高了学生进一步参与和自主学习的兴趣:该倡议利用全球可获取的资源开展研究培训,有效地培养了参与者的研究能力。它成功展示了本科生为医学研究成果做出贡献的潜力,并为学生主导的自立研究项目铺平了道路。尽管在后勤方面存在一些挑战,但该项目为今后的实施提供了宝贵的见解,展示了学生参与有意义、可发表的研究的潜力。
{"title":"A Pilot Project to Promote Research Competency in Medical Students Through Journal Clubs: Mixed Methods Study.","authors":"Mert Karabacak, Zeynep Ozcan, Burak Berksu Ozkara, Zeynep Sude Furkan, Sotirios Bisdas","doi":"10.2196/51173","DOIUrl":"10.2196/51173","url":null,"abstract":"<p><strong>Background: </strong>Undergraduate medical students often lack hands-on research experience and fundamental scientific research skills, limiting their exposure to the practical aspects of scientific investigation. The Cerrahpasa Neuroscience Society introduced a program to address this deficiency and facilitate student-led research.</p><p><strong>Objective: </strong>The primary goal of this initiative was to enhance medical students' research output by enabling them to generate and publish peer-reviewed papers within the framework of this pilot project. The project aimed to provide an accessible, global model for research training through structured journal clubs, mentorship from experienced peers, and resource access.</p><p><strong>Methods: </strong>In January 2022, a total of 30 volunteer students from various Turkish medical schools participated in this course-based undergraduate research experience program. Students self-organized into 2 groups according to their preferred study type: original research or systematic review. Two final-year students with prior research experience led the project, developing training modules using selected materials. The project was implemented entirely online, with participants completing training modules before using their newly acquired theoretical knowledge to perform assigned tasks.</p><p><strong>Results: </strong>Based on student feedback, the project timeline was adjusted to allow for greater flexibility in meeting deadlines. Despite these adjustments, participants successfully completed their tasks, applying the theoretical knowledge they had gained to their respective assignments. As of April 2024, the initiative has culminated in 3 published papers and 3 more under peer review. The project has also seen an increase in student interest in further involvement and self-paced learning.</p><p><strong>Conclusions: </strong>This initiative leverages globally accessible resources for research training, effectively fostering research competency among participants. It has successfully demonstrated the potential for undergraduates to contribute to medical research output and paved the way for a self-sustaining, student-led research program. Despite some logistical challenges, the project provided valuable insights for future implementations, showcasing the potential for students to engage in meaningful, publishable research.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e51173"},"PeriodicalIF":3.2,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11542906/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142559030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A SIMBA CoMICs Initiative to Cocreating and Disseminating Evidence-Based, Peer-Reviewed Short Videos on Social Media: Mixed Methods Prospective Study. SIMBA CoMICs 在社交媒体上共同创作和传播基于证据、经同行评审的短片的倡议:混合方法前瞻性研究。
IF 4.3 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-10-30 DOI: 10.2196/52924
Maiar Elhariry, Kashish Malhotra, Kashish Goyal, Marco Bardus, Punith Kempegowda
<p><strong>Background: </strong>Social media is a powerful platform for disseminating health information, yet it is often riddled with misinformation. Further, few guidelines exist for producing reliable, peer-reviewed content. This study describes a framework for creating and disseminating evidence-based videos on polycystic ovary syndrome (PCOS) and thyroid conditions to improve health literacy and tackle misinformation.</p><p><strong>Objective: </strong>The study aims to evaluate the creation, dissemination, and impact of evidence-based, peer-reviewed short videos on PCOS and thyroid disorders across social media. It also explores the experiences of content creators and assesses audience engagement.</p><p><strong>Methods: </strong>This mixed methods prospective study was conducted between December 2022 and May 2023 and comprised five phases: (1) script generation, (2) video creation, (3) cross-platform publication, (4) process evaluation, and (5) impact evaluation. The SIMBA-CoMICs (Simulation via Instant Messaging for Bedside Application-Combined Medical Information Cines) initiative provides a structured process where medical concepts are simplified and converted to visually engaging videos. The initiative recruited medical students interested in making visually appealing and scientifically accurate videos for social media. The students were then guided to create video scripts based on frequently searched PCOS- and thyroid-related topics. Once experts confirmed the accuracy of the scripts, the medical students produced the videos. The videos were checked by clinical experts and experts with lived experience to ensure clarity and engagement. The SIMBA-CoMICs team then guided the students in editing these videos to fit platform requirements before posting them on TikTok, Instagram, YouTube, and Twitter. Engagement metrics were tracked over 2 months. Content creators were interviewed, and thematic analysis was performed to explore their experiences.</p><p><strong>Results: </strong>The 20 videos received 718 likes, 120 shares, and 54,686 views across all platforms, with TikTok (19,458 views) and Twitter (19,678 views) being the most popular. Engagement increased significantly, with follower growth ranging from 5% on Twitter to 89% on TikTok. Thematic analysis of interviews with 8 out of 38 participants revealed 4 key themes: views on social media, advice for using social media, reasons for participating, and reflections on the project. Content creators highlighted the advantages of social media, such as large outreach (12 references), convenience (10 references), and accessibility to opportunities (7 references). Participants appreciated the nonrestrictive participation criteria, convenience (8 references), and the ability to record from home using prewritten scripts (6 references). Further recommendations to improve the content creation experience included awareness of audience demographics (9 references), sharing content on multiple platforms
背景社交媒体是传播健康信息的一个强大平台,但其中往往充斥着错误信息。此外,在制作可靠的、经过同行评审的内容方面也鲜有指南。本研究描述了一个创建和传播多囊卵巢综合症(PCOS)和甲状腺疾病循证视频的框架,以提高健康素养和应对错误信息:本研究旨在评估社交媒体上关于多囊卵巢综合征(PCOS)和甲状腺疾病的循证同行评审短视频的制作、传播和影响。研究还探讨了内容创作者的经验,并评估了受众的参与度:这项混合方法前瞻性研究在 2022 年 12 月至 2023 年 5 月期间进行,包括五个阶段:(1)脚本生成;(2)视频创作;(3)跨平台发布;(4)过程评估;(5)影响评估。SIMBA-CoMICs(床旁应用即时通讯模拟-组合医学信息视频)计划提供了一个结构化的过程,将医学概念简化并转换成具有视觉吸引力的视频。该计划招募有兴趣为社交媒体制作具有视觉吸引力和科学准确性视频的医学生。然后,指导学生根据经常搜索的多囊卵巢综合症和甲状腺相关主题制作视频脚本。专家确认脚本的准确性后,医学生们就开始制作视频。临床专家和有生活经验的专家对视频进行了检查,以确保清晰度和参与性。随后,SIMBA-CoMICs 团队指导学生编辑这些视频,使其符合平台要求,然后发布到 TikTok、Instagram、YouTube 和 Twitter 上。在两个月的时间里对参与度指标进行了跟踪。对内容创作者进行了访谈,并进行了主题分析,以探讨他们的经验:20 个视频在所有平台上获得了 718 个赞、120 次分享和 54,686 次浏览,其中 TikTok(19,458 次浏览)和 Twitter(19,678 次浏览)最受欢迎。参与度大幅提高,Twitter 追随者增长 5%,TikTok 追随者增长 89%。对 38 位参与者中的 8 位进行的访谈进行了主题分析,发现了 4 个关键主题:对社交媒体的看法、使用社交媒体的建议、参与的原因以及对项目的反思。内容创作者强调了社交媒体的优势,如覆盖面广(12 次提及)、方便(10 次提及)和机会多(7 次提及)。参与者则对无限制的参与标准、便利性(8 次提及)以及在家使用预先写好的脚本进行录制的能力(6 次提及)表示赞赏。关于改善内容创作体验的其他建议包括:了解受众人口统计(9 次引用)、在多个平台上共享内容(5 次引用)以及与组织机构合作(3 次引用):本研究表明,SIMBA CoMICs 计划在培训医学生为社交媒体传播创建准确的多囊卵巢综合症和甲状腺疾病医疗信息方面非常有效。该模式为消除错误信息和提高健康素养提供了一个可扩展的解决方案。
{"title":"A SIMBA CoMICs Initiative to Cocreating and Disseminating Evidence-Based, Peer-Reviewed Short Videos on Social Media: Mixed Methods Prospective Study.","authors":"Maiar Elhariry, Kashish Malhotra, Kashish Goyal, Marco Bardus, Punith Kempegowda","doi":"10.2196/52924","DOIUrl":"10.2196/52924","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Social media is a powerful platform for disseminating health information, yet it is often riddled with misinformation. Further, few guidelines exist for producing reliable, peer-reviewed content. This study describes a framework for creating and disseminating evidence-based videos on polycystic ovary syndrome (PCOS) and thyroid conditions to improve health literacy and tackle misinformation.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;The study aims to evaluate the creation, dissemination, and impact of evidence-based, peer-reviewed short videos on PCOS and thyroid disorders across social media. It also explores the experiences of content creators and assesses audience engagement.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;This mixed methods prospective study was conducted between December 2022 and May 2023 and comprised five phases: (1) script generation, (2) video creation, (3) cross-platform publication, (4) process evaluation, and (5) impact evaluation. The SIMBA-CoMICs (Simulation via Instant Messaging for Bedside Application-Combined Medical Information Cines) initiative provides a structured process where medical concepts are simplified and converted to visually engaging videos. The initiative recruited medical students interested in making visually appealing and scientifically accurate videos for social media. The students were then guided to create video scripts based on frequently searched PCOS- and thyroid-related topics. Once experts confirmed the accuracy of the scripts, the medical students produced the videos. The videos were checked by clinical experts and experts with lived experience to ensure clarity and engagement. The SIMBA-CoMICs team then guided the students in editing these videos to fit platform requirements before posting them on TikTok, Instagram, YouTube, and Twitter. Engagement metrics were tracked over 2 months. Content creators were interviewed, and thematic analysis was performed to explore their experiences.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;The 20 videos received 718 likes, 120 shares, and 54,686 views across all platforms, with TikTok (19,458 views) and Twitter (19,678 views) being the most popular. Engagement increased significantly, with follower growth ranging from 5% on Twitter to 89% on TikTok. Thematic analysis of interviews with 8 out of 38 participants revealed 4 key themes: views on social media, advice for using social media, reasons for participating, and reflections on the project. Content creators highlighted the advantages of social media, such as large outreach (12 references), convenience (10 references), and accessibility to opportunities (7 references). Participants appreciated the nonrestrictive participation criteria, convenience (8 references), and the ability to record from home using prewritten scripts (6 references). Further recommendations to improve the content creation experience included awareness of audience demographics (9 references), sharing content on multiple platforms ","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e52924"},"PeriodicalIF":4.3,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11561432/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142548074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Utilization of, Perceptions on, and Intention to Use AI Chatbots Among Medical Students in China: National Cross-Sectional Study. 中国医学生对人工智能聊天机器人的使用情况、看法和意向:全国横断面研究
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2024-10-28 DOI: 10.2196/57132
Wenjuan Tao, Jinming Yang, Xing Qu

Background: Artificial intelligence (AI) chatbots are poised to have a profound impact on medical education. Medical students, as early adopters of technology and future health care providers, play a crucial role in shaping the future of health care. However, little is known about the utilization of, perceptions on, and intention to use AI chatbots among medical students in China.

Objective: This study aims to explore the utilization of, perceptions on, and intention to use generative AI chatbots among medical students in China, using the Unified Theory of Acceptance and Use of Technology (UTAUT) framework. By conducting a national cross-sectional survey, we sought to identify the key determinants that influence medical students' acceptance of AI chatbots, thereby providing a basis for enhancing their integration into medical education. Understanding these factors is crucial for educators, policy makers, and technology developers to design and implement effective AI-driven educational tools that align with the needs and expectations of future health care professionals.

Methods: A web-based electronic survey questionnaire was developed and distributed via social media to medical students across the country. The UTAUT was used as a theoretical framework to design the questionnaire and analyze the data. The relationship between behavioral intention to use AI chatbots and UTAUT predictors was examined using multivariable regression.

Results: A total of 693 participants were from 57 universities covering 21 provinces or municipalities in China. Only a minority (199/693, 28.72%) reported using AI chatbots for studying, with ChatGPT (129/693, 18.61%) being the most commonly used. Most of the participants used AI chatbots for quickly obtaining medical information and knowledge (631/693, 91.05%) and increasing learning efficiency (594/693, 85.71%). Utilization behavior, social influence, facilitating conditions, perceived risk, and personal innovativeness showed significant positive associations with the behavioral intention to use AI chatbots (all P values were <.05).

Conclusions: Chinese medical students hold positive perceptions toward and high intentions to use AI chatbots, but there are gaps between intention and actual adoption. This highlights the need for strategies to improve access, training, and support and provide peer usage examples to fully harness the potential benefits of chatbot technology.

背景:人工智能(AI)聊天机器人将对医学教育产生深远影响:人工智能(AI)聊天机器人将对医学教育产生深远影响。医学生作为技术的早期采用者和未来的医疗服务提供者,在塑造医疗保健的未来方面发挥着至关重要的作用。然而,人们对中国医学生使用人工智能聊天机器人的情况、看法和意向知之甚少:本研究旨在采用技术接受和使用统一理论(UTAUT)框架,探讨中国医学生对生成式人工智能聊天机器人的使用情况、看法和使用意向。通过开展全国横断面调查,我们试图找出影响医学生接受人工智能聊天机器人的关键决定因素,从而为加强人工智能聊天机器人与医学教育的结合提供依据。了解这些因素对于教育者、政策制定者和技术开发者设计和实施有效的人工智能驱动的教育工具至关重要,这些工具应符合未来医疗专业人员的需求和期望:方法:我们开发了一个基于网络的电子调查问卷,并通过社交媒体向全国各地的医学生发放。UTAUT作为设计问卷和分析数据的理论框架。使用多元回归法研究了使用人工智能聊天机器人的行为意向与UTAUT预测因素之间的关系:共有 693 名参与者来自中国 21 个省市的 57 所高校。只有少数人(199/693,28.72%)表示在学习中使用了人工智能聊天机器人,其中最常用的是 ChatGPT(129/693,18.61%)。大多数参与者使用人工智能聊天机器人来快速获取医疗信息和知识(631/693,91.05%)以及提高学习效率(594/693,85.71%)。使用行为、社会影响、便利条件、感知风险和个人创新性与使用人工智能聊天机器人的行为意向呈显著正相关(所有 P 值均为结论):中国医学生对人工智能聊天机器人持有积极的看法和较高的使用意愿,但在意愿和实际采用之间存在差距。这凸显出需要制定策略来改善获取、培训和支持,并提供同行使用范例,以充分利用聊天机器人技术的潜在优势。
{"title":"Utilization of, Perceptions on, and Intention to Use AI Chatbots Among Medical Students in China: National Cross-Sectional Study.","authors":"Wenjuan Tao, Jinming Yang, Xing Qu","doi":"10.2196/57132","DOIUrl":"10.2196/57132","url":null,"abstract":"<p><strong>Background: </strong>Artificial intelligence (AI) chatbots are poised to have a profound impact on medical education. Medical students, as early adopters of technology and future health care providers, play a crucial role in shaping the future of health care. However, little is known about the utilization of, perceptions on, and intention to use AI chatbots among medical students in China.</p><p><strong>Objective: </strong>This study aims to explore the utilization of, perceptions on, and intention to use generative AI chatbots among medical students in China, using the Unified Theory of Acceptance and Use of Technology (UTAUT) framework. By conducting a national cross-sectional survey, we sought to identify the key determinants that influence medical students' acceptance of AI chatbots, thereby providing a basis for enhancing their integration into medical education. Understanding these factors is crucial for educators, policy makers, and technology developers to design and implement effective AI-driven educational tools that align with the needs and expectations of future health care professionals.</p><p><strong>Methods: </strong>A web-based electronic survey questionnaire was developed and distributed via social media to medical students across the country. The UTAUT was used as a theoretical framework to design the questionnaire and analyze the data. The relationship between behavioral intention to use AI chatbots and UTAUT predictors was examined using multivariable regression.</p><p><strong>Results: </strong>A total of 693 participants were from 57 universities covering 21 provinces or municipalities in China. Only a minority (199/693, 28.72%) reported using AI chatbots for studying, with ChatGPT (129/693, 18.61%) being the most commonly used. Most of the participants used AI chatbots for quickly obtaining medical information and knowledge (631/693, 91.05%) and increasing learning efficiency (594/693, 85.71%). Utilization behavior, social influence, facilitating conditions, perceived risk, and personal innovativeness showed significant positive associations with the behavioral intention to use AI chatbots (all P values were <.05).</p><p><strong>Conclusions: </strong>Chinese medical students hold positive perceptions toward and high intentions to use AI chatbots, but there are gaps between intention and actual adoption. This highlights the need for strategies to improve access, training, and support and provide peer usage examples to fully harness the potential benefits of chatbot technology.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e57132"},"PeriodicalIF":3.2,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11533383/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142509604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
JMIR Medical Education
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1