首页 > 最新文献

JMIR Medical Education最新文献

英文 中文
AI Competency: Current State and Challenges. 人工智能能力:现状与挑战。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-03-03 DOI: 10.2196/86686
Sian Tsuei

Unlabelled: As artificial intelligence (AI) develops, the medical education community has begun defining the relevant forms of competency. Many experts emphasize the importance of optimizing AI tools' output or understanding the relevant technical and normative considerations around using AI tools. A recent publication in this journal showed that optimizing instructions for large language models may yield diminishing returns as such tools improve. This suggests the need for a new competency-one that focuses on choosing the appropriate AI tools. I briefly summarize the current competency domains and examples to contextualize the current state of AI competency development, highlighting the need for further synthesis. I then introduce a hierarchical framework of competencies that might assist with priority-setting around subsequent competency development work. It consists of cognitive, operational, and meta-AI domains, which respectively correspond with the knowledge around understanding, using, and choosing AI tools. The final section describes the potential challenges associated with the development of AI competency. These include traditional concerns around competency-based medical education: deciding whether and which competencies are meaningful for measuring the targets of interest; adjusting the relevant measurements to reflect the necessary temporal and institutional context; and setting up the relevant organizational support to encourage measurement of competency. This section also discusses the challenges of developing the relevant performance indicators for AI tools across different clinical contexts. Such indicators will be necessary for guiding the choice of AI tools for the clinical context, but medical educators may not have the skills to develop them. In addition to identifying potential sources for relevant indicators, the medical education community may shape physicians' norms of practice to drive the AI industry into producing the relevant indicators. The potential for physicians to incur higher medical liability from poor choice of AI may lead them to demand more nuanced performance indicators from AI suppliers. Physicians are also in a position to do so, since the competitive AI market may provide them more bargaining power.

未标记:随着人工智能(AI)的发展,医学教育界已经开始定义相关的能力形式。许多专家强调优化人工智能工具输出或理解使用人工智能工具的相关技术和规范考虑的重要性。该杂志最近发表的一篇文章表明,优化大型语言模型的指令可能会随着工具的改进而产生递减的回报。这表明需要一种新的能力——一种专注于选择合适的人工智能工具的能力。我简要地总结了当前的能力领域和例子,以背景化人工智能能力发展的现状,强调了进一步综合的必要性。然后,我介绍了一个能力的层次框架,它可能有助于围绕随后的能力开发工作设置优先级。它由认知、操作和元AI领域组成,分别对应于理解、使用和选择AI工具的知识。最后一部分描述了与人工智能能力发展相关的潜在挑战。这些问题包括对基于能力的医学教育的传统关注:决定哪些能力对于衡量感兴趣的目标是否有意义;调整有关的衡量标准,以反映必要的时间和制度背景;并建立相关的组织支持来鼓励能力的测量。本节还讨论了在不同临床背景下为人工智能工具制定相关绩效指标所面临的挑战。这些指标对于指导临床环境下人工智能工具的选择是必要的,但医学教育工作者可能没有开发这些指标的技能。除了确定相关指标的潜在来源外,医学教育界还可以塑造医生的实践规范,以推动人工智能行业制定相关指标。医生可能会因为人工智能的选择不当而承担更高的医疗责任,这可能会导致他们要求人工智能供应商提供更细致入微的绩效指标。医生也可以这样做,因为竞争激烈的人工智能市场可能会为他们提供更多的议价能力。
{"title":"AI Competency: Current State and Challenges.","authors":"Sian Tsuei","doi":"10.2196/86686","DOIUrl":"10.2196/86686","url":null,"abstract":"<p><strong>Unlabelled: </strong>As artificial intelligence (AI) develops, the medical education community has begun defining the relevant forms of competency. Many experts emphasize the importance of optimizing AI tools' output or understanding the relevant technical and normative considerations around using AI tools. A recent publication in this journal showed that optimizing instructions for large language models may yield diminishing returns as such tools improve. This suggests the need for a new competency-one that focuses on choosing the appropriate AI tools. I briefly summarize the current competency domains and examples to contextualize the current state of AI competency development, highlighting the need for further synthesis. I then introduce a hierarchical framework of competencies that might assist with priority-setting around subsequent competency development work. It consists of cognitive, operational, and meta-AI domains, which respectively correspond with the knowledge around understanding, using, and choosing AI tools. The final section describes the potential challenges associated with the development of AI competency. These include traditional concerns around competency-based medical education: deciding whether and which competencies are meaningful for measuring the targets of interest; adjusting the relevant measurements to reflect the necessary temporal and institutional context; and setting up the relevant organizational support to encourage measurement of competency. This section also discusses the challenges of developing the relevant performance indicators for AI tools across different clinical contexts. Such indicators will be necessary for guiding the choice of AI tools for the clinical context, but medical educators may not have the skills to develop them. In addition to identifying potential sources for relevant indicators, the medical education community may shape physicians' norms of practice to drive the AI industry into producing the relevant indicators. The potential for physicians to incur higher medical liability from poor choice of AI may lead them to demand more nuanced performance indicators from AI suppliers. Physicians are also in a position to do so, since the competitive AI market may provide them more bargaining power.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e86686"},"PeriodicalIF":3.2,"publicationDate":"2026-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12978887/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147436045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Prescription Support Practice for Pharmacy Students: Pre-Post Educational Intervention Study. 药学专业学生的处方支持实践:教育干预前后研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-03-02 DOI: 10.2196/79545
Fuka Aizawa, Kenta Yagi, Tsukasa Higashionna, Hirofumi Hamano, Shimon Takahashi, Yoshito Zamami, Kazuaki Shinomiya, Takahiro Niimura, Mitsuhiro Goda, Kei Kawada, Keisuke Ishizawa

Background: In the field of team-based care, pharmacists are vital for optimizing medication therapy. However, many medical professionals lack the opportunity to learn how to propose prescription changes with precision.

Objective: This study aimed to address this knowledge gap by developing and assessing a new educational program for pharmacy students focused on prescription support and interprofessional collaboration.

Methods: We recruited 191 fifth-year pharmaceutical students during the 2022-2024 academic years. The program featured a 7-day intensive curriculum that included learning how to assist with prescriptions, analyzing clinical data, and engaging in role-playing exercises. A web-based questionnaire and a paper test were used to evaluate students' awareness and knowledge both before and after the program. Statistical analyses were performed to verify the significance of changes; we utilized the Wilcoxon signed-rank test for the ordinal data derived from the specific behavioral objectives and 2-tailed paired t tests for the interval data from the knowledge tests. The magnitude of change was quantified using r for Wilcoxon tests and Cohen dz for 2-tailed t tests, with 95% CI calculated to ensure the stability and reliability of the observed results.

Results: Analysis of the primary outcome specific behavioral objectives revealed statistically significant effects across all items (Wilcoxon signed-rank test; P<.001). Effect sizes (r=0.505-0.835) ranged from moderate to large, with particularly large effects observed in identifying contents issue (r=0.835, 95% CI 0.126-0.330; P<.001). Knowledge test scores showed significant improvement in the following 3 subjects: pharmacology (r=-0.504, 95% CI -0.215 to 0.127; P<.001), organic chemistry (r=0.254, 95% CI -0.148 to -0.193; P=.004), and communication (r=0.221, 95% CI -0.151 to -0.190; P=.01). No significant changes were observed in pathology or pharmacokinetics.

Conclusions: This program provides strong evidence that practical, hands-on learning with hospital pharmacists helps improve pharmacy students' professional skills and optimize pharmaceutical therapies in interprofessional care. By teaching pharmacists to effectively propose prescription changes, the program equips them to become integral members of interprofessional care, ultimately leading to optimized pharmaceutical care for patients.

背景:在团队护理领域,药师是优化药物治疗的关键。然而,许多医疗专业人员缺乏学习如何精确地提出处方变更的机会。目的:本研究旨在通过开发和评估一个新的以处方支持和跨专业合作为重点的药学学生教育计划来解决这一知识差距。方法:我们在2022-2024学年招募了191名药学五年级学生。该计划的特色是为期7天的强化课程,包括学习如何协助处方,分析临床数据,以及参与角色扮演练习。采用基于网络的问卷和纸质测试来评估学生在课程前后的意识和知识。通过统计学分析验证变化的显著性;我们对来自特定行为目标的有序数据使用Wilcoxon符号秩检验,对来自知识测试的区间数据使用双尾配对t检验。Wilcoxon检验用r量化变化幅度,双尾t检验用Cohen dz量化变化幅度,并计算95% CI以确保观察结果的稳定性和可靠性。结果:对主要结局、具体行为目标的分析显示,在所有项目上都有统计学上显著的效果(Wilcoxon sign -rank test; p)。结论:该项目提供了强有力的证据,证明与医院药剂师一起实践、动手学习有助于提高药学学生的专业技能,并优化跨专业护理中的药物治疗。通过教授药剂师有效地提出处方变更,该计划使他们成为跨专业护理的不可或缺的成员,最终为患者提供优化的药物护理。
{"title":"Prescription Support Practice for Pharmacy Students: Pre-Post Educational Intervention Study.","authors":"Fuka Aizawa, Kenta Yagi, Tsukasa Higashionna, Hirofumi Hamano, Shimon Takahashi, Yoshito Zamami, Kazuaki Shinomiya, Takahiro Niimura, Mitsuhiro Goda, Kei Kawada, Keisuke Ishizawa","doi":"10.2196/79545","DOIUrl":"10.2196/79545","url":null,"abstract":"<p><strong>Background: </strong>In the field of team-based care, pharmacists are vital for optimizing medication therapy. However, many medical professionals lack the opportunity to learn how to propose prescription changes with precision.</p><p><strong>Objective: </strong>This study aimed to address this knowledge gap by developing and assessing a new educational program for pharmacy students focused on prescription support and interprofessional collaboration.</p><p><strong>Methods: </strong>We recruited 191 fifth-year pharmaceutical students during the 2022-2024 academic years. The program featured a 7-day intensive curriculum that included learning how to assist with prescriptions, analyzing clinical data, and engaging in role-playing exercises. A web-based questionnaire and a paper test were used to evaluate students' awareness and knowledge both before and after the program. Statistical analyses were performed to verify the significance of changes; we utilized the Wilcoxon signed-rank test for the ordinal data derived from the specific behavioral objectives and 2-tailed paired t tests for the interval data from the knowledge tests. The magnitude of change was quantified using r for Wilcoxon tests and Cohen dz for 2-tailed t tests, with 95% CI calculated to ensure the stability and reliability of the observed results.</p><p><strong>Results: </strong>Analysis of the primary outcome specific behavioral objectives revealed statistically significant effects across all items (Wilcoxon signed-rank test; P<.001). Effect sizes (r=0.505-0.835) ranged from moderate to large, with particularly large effects observed in identifying contents issue (r=0.835, 95% CI 0.126-0.330; P<.001). Knowledge test scores showed significant improvement in the following 3 subjects: pharmacology (r=-0.504, 95% CI -0.215 to 0.127; P<.001), organic chemistry (r=0.254, 95% CI -0.148 to -0.193; P=.004), and communication (r=0.221, 95% CI -0.151 to -0.190; P=.01). No significant changes were observed in pathology or pharmacokinetics.</p><p><strong>Conclusions: </strong>This program provides strong evidence that practical, hands-on learning with hospital pharmacists helps improve pharmacy students' professional skills and optimize pharmaceutical therapies in interprofessional care. By teaching pharmacists to effectively propose prescription changes, the program equips them to become integral members of interprofessional care, ultimately leading to optimized pharmaceutical care for patients.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e79545"},"PeriodicalIF":3.2,"publicationDate":"2026-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12954723/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147345448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Application of AI-Generated Content in Medical Education: Systematic Review of the Impact on Critical Thinking Abilities of Medical Students. 人工智能生成内容在医学教育中的应用:对医学生批判性思维能力影响的系统回顾
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-27 DOI: 10.2196/79939
Jinlei Li, Fen Ai, Jueyan Wang, Bingxin Cheng, Yu Li, Zhen Chen
<p><strong>Background: </strong>With the rapid development of artificial intelligence technology, artificial intelligence-generated content (AIGC) is increasingly widely applied in the field of medical education. Large language models, such as ChatGPT, are a prominent type of AIGC technology. Critical thinking is a core ability in medical education, but the impact of AIGC technology on the critical thinking ability of medical students remains unclear. Medical students are at a crucial stage in cultivating critical thinking, and the intervention of AIGC technology may have a profound impact on this process.</p><p><strong>Objective: </strong>This study aims to systematically review the impact of AIGC technology on the complex mechanisms affecting medical students' critical thinking abilities and build a corresponding strategic framework. The findings are intended to provide theoretical support and practical guidance for applying AIGC in medical education.</p><p><strong>Methods: </strong>This study followed 2020 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, with the retrieval scope limited to English studies published between November 2022 and June 2025. Through the PubMed database, combined with the search methods of subject terms and free words, relevant studies involving the impact of AIGC on the critical thinking of medical students were screened for using keywords such as "AIGC," "medical students," and "critical thinking." Two independent reviewers screened and evaluated the literature, and ultimately conducted a qualitative analysis based on the common themes extracted from the literature.</p><p><strong>Results: </strong>AIGC technology in medical education is 2-fold. First, AIGC's powerful information capabilities provide abundant learning resources and efficient tools. This accelerates knowledge acquisition and broadens learning scope. Second, overreliance on AIGC may lead to mental inertia, weaken critical thinking skills, and cause academic integrity issues among students. Research has found that strategies such as customized AIGC tools, virtual standardized patients, new models of resource integration, and proactive assessment of AI limitations can effectively make up for the deficiencies of AIGC in cultivating high-level critical thinking, helping medical students maintain and enhance their critical thinking and problem-solving abilities.</p><p><strong>Conclusions: </strong>AIGC technology application in medical education needs to carefully weigh the pros and cons. By optimizing the design and usage of AIGC tools and combining them with the guidance and supervision of educators, they can be transformed into powerful tools for promoting the development of critical thinking among medical students. Future research should further expand the scope of study, optimize research methods, pay attention to individual differences, track long-term effects, and deeply explore the influence of ethical and cult
背景:随着人工智能技术的快速发展,人工智能生成内容(AIGC)在医学教育领域的应用越来越广泛。大型语言模型,如ChatGPT,是AIGC技术的一种突出类型。批判性思维是医学教育的核心能力,但AIGC技术对医学生批判性思维能力的影响尚不清楚。医学生正处于批判性思维培养的关键阶段,AIGC技术的介入可能会对这一过程产生深远的影响。目的:本研究旨在系统回顾AIGC技术对医学生批判性思维能力复杂机制的影响,并构建相应的策略框架。研究结果旨在为AIGC在医学教育中的应用提供理论支持和实践指导。方法:本研究遵循2020年PRISMA(系统评价和荟萃分析首选报告项目)指南,检索范围限于2022年11月至2025年6月期间发表的英文研究。通过PubMed数据库,结合主题词和自由词的搜索方法,使用“AIGC”、“医学生”、“批判性思维”等关键词,筛选涉及AIGC对医学生批判性思维影响的相关研究。两位独立的审稿人对文献进行筛选和评估,并最终根据从文献中提取的共同主题进行定性分析。结果:AIGC技术在医学教育中具有双重作用。首先,AIGC强大的信息能力提供了丰富的学习资源和高效的工具。这加快了知识获取,拓宽了学习范围。其次,过度依赖AIGC可能会导致心理惰性,削弱批判性思维能力,并导致学生的学术诚信问题。研究发现,定制化AIGC工具、虚拟标准化患者、资源整合新模式、主动评估AI局限性等策略可以有效弥补AIGC在培养高水平批判性思维方面的不足,帮助医学生保持和增强批判性思维和解决问题的能力。结论:AIGC技术在医学教育中的应用需要慎重权衡利弊,通过优化AIGC工具的设计和使用,并结合教育者的指导和监督,使其成为促进医学生批判性思维发展的有力工具。未来的研究应进一步扩大研究范围,优化研究方法,关注个体差异,跟踪长期效果,深入探讨伦理和文化因素的影响,更全面地评估AIGC技术在医学教育中的应用潜力和挑战。
{"title":"Application of AI-Generated Content in Medical Education: Systematic Review of the Impact on Critical Thinking Abilities of Medical Students.","authors":"Jinlei Li, Fen Ai, Jueyan Wang, Bingxin Cheng, Yu Li, Zhen Chen","doi":"10.2196/79939","DOIUrl":"10.2196/79939","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;With the rapid development of artificial intelligence technology, artificial intelligence-generated content (AIGC) is increasingly widely applied in the field of medical education. Large language models, such as ChatGPT, are a prominent type of AIGC technology. Critical thinking is a core ability in medical education, but the impact of AIGC technology on the critical thinking ability of medical students remains unclear. Medical students are at a crucial stage in cultivating critical thinking, and the intervention of AIGC technology may have a profound impact on this process.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aims to systematically review the impact of AIGC technology on the complex mechanisms affecting medical students' critical thinking abilities and build a corresponding strategic framework. The findings are intended to provide theoretical support and practical guidance for applying AIGC in medical education.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;This study followed 2020 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, with the retrieval scope limited to English studies published between November 2022 and June 2025. Through the PubMed database, combined with the search methods of subject terms and free words, relevant studies involving the impact of AIGC on the critical thinking of medical students were screened for using keywords such as \"AIGC,\" \"medical students,\" and \"critical thinking.\" Two independent reviewers screened and evaluated the literature, and ultimately conducted a qualitative analysis based on the common themes extracted from the literature.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;AIGC technology in medical education is 2-fold. First, AIGC's powerful information capabilities provide abundant learning resources and efficient tools. This accelerates knowledge acquisition and broadens learning scope. Second, overreliance on AIGC may lead to mental inertia, weaken critical thinking skills, and cause academic integrity issues among students. Research has found that strategies such as customized AIGC tools, virtual standardized patients, new models of resource integration, and proactive assessment of AI limitations can effectively make up for the deficiencies of AIGC in cultivating high-level critical thinking, helping medical students maintain and enhance their critical thinking and problem-solving abilities.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;AIGC technology application in medical education needs to carefully weigh the pros and cons. By optimizing the design and usage of AIGC tools and combining them with the guidance and supervision of educators, they can be transformed into powerful tools for promoting the development of critical thinking among medical students. Future research should further expand the scope of study, optimize research methods, pay attention to individual differences, track long-term effects, and deeply explore the influence of ethical and cult","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e79939"},"PeriodicalIF":3.2,"publicationDate":"2026-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12954715/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147345398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Real-World Impact and Educational Effectiveness of an AI-Powered Medical History-Taking System: Retrospective Propensity Score-Matched Cohort Study. 人工智能驱动的病史采集系统的现实世界影响和教育有效性:回顾性倾向评分匹配队列研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-24 DOI: 10.2196/89367
Yang Liu, Yiying Zhu, Weishan Zhang, Xian Lu, Liping Wu, Minghui Yue, Oudong Xia, Chujun Shi
<p><strong>Background: </strong>Medical history-taking is a core clinical skill; yet, traditional teaching methods face challenges. We developed an artificial intelligence-powered medical history-taking training and evaluation system (AMTES) and established its technical feasibility as an extracurricular resource. Evidence on whether such tools improve learning outcomes when voluntarily embedded in routine curricula remains limited.</p><p><strong>Objective: </strong>This study aimed to evaluate the real-world educational effectiveness of AMTES as an opt-in extracurricular tool and examine whether learning gains vary by practice patterns and baseline academic ability.</p><p><strong>Methods: </strong>We conducted a retrospective cohort study of the 2024-2025 Diagnostics course cohort (N=478) at Shantou University Medical College, China, using total population sampling. Students were categorized as AMTES users (n=205, 42.9%; ≥1 sessions) and nonusers (n=273, 57.1%) based on their voluntary extracurricular adoption of the system during the month preceding a high-stakes final practical skills examination. To address selection bias, we performed 1:1 propensity score matching via logistic regression using age, sex, and 3 previous academic scores as covariates. The average treatment effect on the treated for final examination score (0-70) was estimated with paired t tests, and robustness to unobserved confounding was assessed via Rosenbaum sensitivity analysis. Among matched users, practice patterns were identified using K-means clustering on log-derived features, with cluster differences compared using Mann-Whitney U tests. Subsequently, we explored aptitude-treatment interaction by testing the interaction between practice intensity and baseline ability using linear and logistic regression models.</p><p><strong>Results: </strong>Propensity score matching yielded 157 matched pairs (n=314) with excellent covariate balance (|standardized mean difference|<0.1). In the matched cohort, the users outperformed nonusers by 3% (average treatment effect on the treated=2.09, 95% CI 0.75-3.42; P=.002). This finding was robust to weak unmeasured confounding (Rosenbaum Γ=1.23). Among users (N=157), cluster analysis of usage logs revealed a low-intensity group (74/157, 47.1%) and a high-intensity group (83/157, 52.9%). The 2 groups reflected differences in both practice quantity and quality. However, the added efforts did not translate into higher scores (mean difference=1.6 points, 95% CI -0.5 to 3.6) or excellence probability (risk difference=7.7 percentage points, 95% CI -5.0 to 20.5). Exploratory aptitude-treatment interaction analyses suggested ability-dependent effects for excellence rate (β<sub>3</sub>=1.461; P=.04) and marginally for final score (β<sub>3</sub>=2.58; P=.07), but not for pass rate (P=.94).</p><p><strong>Conclusions: </strong>Building upon previous technical validation, this study contributes real-world effectiveness evidence by evaluating AMTES a
背景:病史记录是临床的核心技能;然而,传统的教学方法面临着挑战。我们开发了人工智能驱动的病史采集培训和评估系统(AMTES),并确定了其作为课外资源的技术可行性。关于自愿将这些工具纳入日常课程是否能改善学习效果的证据仍然有限。目的:本研究旨在评估AMTES作为一种可选择的课外工具的实际教育效果,并检查学习收益是否因实践模式和基线学术能力而变化。方法:采用总体抽样方法,对汕头大学医学院2024-2025年诊断学课程队列(N=478)进行回顾性队列研究。根据学生在高风险的期末实践技能考试前一个月自愿课外使用该系统的情况,将学生分为AMTES使用者(n=205, 42.9%;≥1次)和非使用者(n=273, 57.1%)。为了解决选择偏差,我们使用年龄、性别和3个以前的学业成绩作为协变量,通过逻辑回归进行了1:1的倾向评分匹配。对期末考试分数(0-70)的平均治疗效果采用配对t检验估计,并通过Rosenbaum敏感性分析评估对未观察到的混杂的稳健性。在匹配的用户中,使用对数衍生特征的K-means聚类识别实践模式,使用Mann-Whitney U检验比较聚类差异。随后,我们利用线性和逻辑回归模型检验了练习强度和基线能力之间的相互作用,从而探讨了能力-治疗的相互作用。结果:倾向得分匹配产生157对匹配对(n=314),具有良好的协变量平衡(|标准化平均差|3=1.461;P= 0.04),最终得分(β3=2.58; P= 0.07),但通配率(P= 0.94)不佳。结论:基于先前的技术验证,本研究通过评估AMTES作为真实的高基线课程中的自愿课外补充,提供了现实世界的有效性证据。与之前的研究不同的是,这些研究关注的是技术可行性或短期对照试验,自愿课外使用AMTES与总结历史记录性能的适度但有意义的改善有关。探索性分析表明,更密集参与的附加价值可能受到基线学术能力的调节。这些发现支持了人工智能辅助训练的可扩展性,并为精确导向的教学设计提供了信息。
{"title":"Real-World Impact and Educational Effectiveness of an AI-Powered Medical History-Taking System: Retrospective Propensity Score-Matched Cohort Study.","authors":"Yang Liu, Yiying Zhu, Weishan Zhang, Xian Lu, Liping Wu, Minghui Yue, Oudong Xia, Chujun Shi","doi":"10.2196/89367","DOIUrl":"10.2196/89367","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Medical history-taking is a core clinical skill; yet, traditional teaching methods face challenges. We developed an artificial intelligence-powered medical history-taking training and evaluation system (AMTES) and established its technical feasibility as an extracurricular resource. Evidence on whether such tools improve learning outcomes when voluntarily embedded in routine curricula remains limited.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to evaluate the real-world educational effectiveness of AMTES as an opt-in extracurricular tool and examine whether learning gains vary by practice patterns and baseline academic ability.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We conducted a retrospective cohort study of the 2024-2025 Diagnostics course cohort (N=478) at Shantou University Medical College, China, using total population sampling. Students were categorized as AMTES users (n=205, 42.9%; ≥1 sessions) and nonusers (n=273, 57.1%) based on their voluntary extracurricular adoption of the system during the month preceding a high-stakes final practical skills examination. To address selection bias, we performed 1:1 propensity score matching via logistic regression using age, sex, and 3 previous academic scores as covariates. The average treatment effect on the treated for final examination score (0-70) was estimated with paired t tests, and robustness to unobserved confounding was assessed via Rosenbaum sensitivity analysis. Among matched users, practice patterns were identified using K-means clustering on log-derived features, with cluster differences compared using Mann-Whitney U tests. Subsequently, we explored aptitude-treatment interaction by testing the interaction between practice intensity and baseline ability using linear and logistic regression models.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Propensity score matching yielded 157 matched pairs (n=314) with excellent covariate balance (|standardized mean difference|&lt;0.1). In the matched cohort, the users outperformed nonusers by 3% (average treatment effect on the treated=2.09, 95% CI 0.75-3.42; P=.002). This finding was robust to weak unmeasured confounding (Rosenbaum Γ=1.23). Among users (N=157), cluster analysis of usage logs revealed a low-intensity group (74/157, 47.1%) and a high-intensity group (83/157, 52.9%). The 2 groups reflected differences in both practice quantity and quality. However, the added efforts did not translate into higher scores (mean difference=1.6 points, 95% CI -0.5 to 3.6) or excellence probability (risk difference=7.7 percentage points, 95% CI -5.0 to 20.5). Exploratory aptitude-treatment interaction analyses suggested ability-dependent effects for excellence rate (β&lt;sub&gt;3&lt;/sub&gt;=1.461; P=.04) and marginally for final score (β&lt;sub&gt;3&lt;/sub&gt;=2.58; P=.07), but not for pass rate (P=.94).&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;Building upon previous technical validation, this study contributes real-world effectiveness evidence by evaluating AMTES a","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e89367"},"PeriodicalIF":3.2,"publicationDate":"2026-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12976603/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147285519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
From Realism to Learner Engagement: Rethinking Fidelity in Simulation-Based Education. 从现实主义到学习者参与:重新思考模拟教育中的保真度。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-23 DOI: 10.2196/84684
Julien Pico, Jean-Noel Evain, Christina Aron, Gilles Martin, Ilian Cruz-Panesso, Leonida-Mihai Georgescu, Issam Tanoubi

Unlabelled: Simulation has become an essential pedagogical tool in health professions education, traditionally valued for its ability to approximate clinical practice. Higher simulation fidelity is often assumed to directly enhance learner engagement and improve educational outcomes; however, this assumption oversimplifies a complex relationship. Fidelity is multidimensional, encompassing physical, emotional, and contextual dimensions, as well as qualitative and quantitative considerations, each influencing learners' perception of realism in distinct ways. Engagement is shaped not only by these dimensions of fidelity but also by intrinsic factors such as motivation, prior experience, stress, and emotional resilience, and by extrinsic factors including instructional design, facilitation, debriefing, and psychological safety. A central mediator in this process is the fiction contract, an implicit agreement that enables learners to suspend disbelief and engage authentically despite inherent limitations in realism. Technological sophistication alone does not necessarily translate into greater educational impact. Rather, fidelity should be intentionally aligned with learning objectives: advanced patient simulators may support procedural training, standardized patients may enhance communication skills, and task trainers may optimize focused psychomotor practice. This viewpoint advocates for a goal-oriented, multimodal approach that redefines high-fidelity simulation not as the pursuit of maximum realism, but as the deliberate alignment of fidelity with pedagogy to optimize learner engagement and educational effectiveness.

未标记:模拟已成为卫生专业教育中必不可少的教学工具,传统上因其接近临床实践的能力而受到重视。更高的模拟保真度通常被认为可以直接提高学习者的参与度并改善教育成果;然而,这种假设过于简化了复杂的关系。保真度是多维度的,包括身体、情感和情境维度,以及定性和定量考虑,每一个都以不同的方式影响学习者对现实主义的感知。参与不仅受到这些忠诚维度的影响,还受到内在因素的影响,如动机、先前经验、压力和情绪弹性,以及外在因素的影响,包括教学设计、促进、汇报和心理安全。在这个过程中,一个中心调解人是小说契约,这是一种隐含的协议,它使学习者能够抛开现实主义固有的局限性,暂停怀疑并真实地参与其中。技术的复杂性本身并不一定转化为更大的教育影响。相反,保真度应该有意识地与学习目标保持一致:先进的病人模拟器可以支持程序性训练,标准化的病人可以提高沟通技巧,任务训练员可以优化专注的精神运动练习。这种观点提倡以目标为导向的多模态方法,重新定义高保真仿真,而不是追求最大的真实感,而是将保真度与教学法相结合,以优化学习者的参与度和教育效率。
{"title":"From Realism to Learner Engagement: Rethinking Fidelity in Simulation-Based Education.","authors":"Julien Pico, Jean-Noel Evain, Christina Aron, Gilles Martin, Ilian Cruz-Panesso, Leonida-Mihai Georgescu, Issam Tanoubi","doi":"10.2196/84684","DOIUrl":"10.2196/84684","url":null,"abstract":"<p><strong>Unlabelled: </strong>Simulation has become an essential pedagogical tool in health professions education, traditionally valued for its ability to approximate clinical practice. Higher simulation fidelity is often assumed to directly enhance learner engagement and improve educational outcomes; however, this assumption oversimplifies a complex relationship. Fidelity is multidimensional, encompassing physical, emotional, and contextual dimensions, as well as qualitative and quantitative considerations, each influencing learners' perception of realism in distinct ways. Engagement is shaped not only by these dimensions of fidelity but also by intrinsic factors such as motivation, prior experience, stress, and emotional resilience, and by extrinsic factors including instructional design, facilitation, debriefing, and psychological safety. A central mediator in this process is the fiction contract, an implicit agreement that enables learners to suspend disbelief and engage authentically despite inherent limitations in realism. Technological sophistication alone does not necessarily translate into greater educational impact. Rather, fidelity should be intentionally aligned with learning objectives: advanced patient simulators may support procedural training, standardized patients may enhance communication skills, and task trainers may optimize focused psychomotor practice. This viewpoint advocates for a goal-oriented, multimodal approach that redefines high-fidelity simulation not as the pursuit of maximum realism, but as the deliberate alignment of fidelity with pedagogy to optimize learner engagement and educational effectiveness.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e84684"},"PeriodicalIF":3.2,"publicationDate":"2026-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12928686/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147277252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Design and Evaluation of an Online Continuing Medical Education App for Medical Professionals in China: Quantitative Study. 中国医学专业人员在线继续医学教育应用程序的设计与评价:定量研究
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-23 DOI: 10.2196/76299
Xu Zhang, Xianying He, Yuntian Chu, Dongqing Liu, Minzhao Lyu, Weiyi Wang, Haotian Chen, Meihao Ji, Fangfang Cui, Jie Zhao
<p><strong>Background: </strong>As an emerging delivery mode of education, online continuing medical education (CME) increases the accessibility of high-quality medical training for professionals and students in China. Guoyuan (meaning "nationwide" in Chinese) is an online CME platform delivered via a mobile app and operated by the National Telemedicine Center of China since 2018, serving as an illustrative case of mobile online CME implementation.</p><p><strong>Objective: </strong>We identified trends in the adoption and usage of the Guoyuan mobile online CME platform from 2018 to 2023 and provided evidence for the application and optimization of online CME.</p><p><strong>Methods: </strong>We analyzed yearly usage data of the Guoyuan mobile app (The First Affiliated Hospital of Zhengzhou University) in 2018-2023 and collected surveys on the satisfaction and recognition of competency enhancement in online CME in each connected hospital in 2023. Using the IBM SPSS, the nonparametric Kruskal-Wallis H test was used to compare attendance across different disciplines, followed by post hoc pairwise comparisons for course types with significant differences and ordinal logistic regression analysis to examine factors influencing satisfaction with the online CME system and perceived competency enhancement among invited doctors.</p><p><strong>Results: </strong>From 2018 to 2023, Guoyuan had 94,537 registered trainees, 1672 published course videos, and 1,878,437 attendances. Attendance was higher for courses in ophthalmology, otolaryngology, and pathology than in other disciplines (median attendance 610, IQR 105-2055 vs 283, IQR 106-690 participants). Based on a sample size of 245 participants, ordinal regression analysis showed that discipline category, professional title, and working years significantly influenced satisfaction. General practitioners showed lower overall satisfaction than internal medicine doctors (odds ratio [OR] 0.323, 95% CI 0.110-0.948; OR 0.251, 95% CI 0.087-0.729; and OR 0.196, 95% CI 0.066-0.585; P=.04; P=.01; P=.003). Junior titles reported higher audio-visual clarity (OR 3.151, 95% CI 1.178-8.427; P=.02) and process satisfaction (OR 4.939, 95% CI 1.674-14.576; P=.004). More experienced doctors had higher system usability (OR 1.102, 95% CI 1.012-1.200; P=.03) and process satisfaction (OR 1.141, 95% CI 1.044-1.247; P=.003). Recognition of online CME's benefits was influenced by multiple factors. Greater clinical experience positively predicted recognition of clinical use (OR 1.106, 95% CI 1.004-1.218; P=.04), while an inverse association was observed with age (OR 0.894, 95% CI 0.802-0.996; P=.04). For research-related benefits, positive predictors included discipline category in obstetrics and gynecology compared to internal medicine (OR 6.217, 95% CI 1.236-31.258; P=.03) and junior professional title (OR 3.791, 95% CI 1.231-11.673; P=.02), whereas intensive care unit was a negative predictor compared to internal medicine (OR 0.111,
背景:作为一种新兴的教育交付模式,在线继续医学教育(CME)增加了中国专业人员和学生获得高质量医学培训的可及性。国远是中国国家远程医疗中心自2018年起运营的移动端在线CME平台,是中国移动端在线CME实施的示范案例。目的:了解2018 - 2023年国远移动在线CME平台的采用和使用趋势,为在线CME的应用和优化提供依据。方法:分析2018-2023年郑州大学第一附属医院国远移动app的年度使用数据,收集各连接医院2023年对在线继续教育能力提升的满意度和认知度调查。采用IBM SPSS软件,采用非参数Kruskal-Wallis H检验比较不同学科的出诊情况,随后对具有显著差异的课程类型进行事后两两比较,并对受邀医生对在线继续教育系统满意度和感知能力提升的影响因素进行有序逻辑回归分析。结果:2018年至2023年,国元注册学员94537人,发布课程视频1672段,听课人数1878437人次。眼科、耳鼻喉科和病理学课程的出勤率高于其他学科(平均出勤率为610人,IQR 105-2055人对283人,IQR 106-690人)。基于245名参与者的样本,有序回归分析显示学科类别、职称和工作年限对满意度有显著影响。全科医生的总体满意度低于内科医生(比值比[OR] 0.323, 95% CI 0.110-0.948; OR 0.251, 95% CI 0.087-0.729; OR 0.196, 95% CI 0.066-0.585; P= 0.04; P= 0.01; P= 0.003)。初级职称报告更高的视听清晰度(OR 3.151, 95% CI 1.178-8.427; P= 0.02)和过程满意度(OR 4.939, 95% CI 1.674-14.576; P= 0.004)。经验丰富的医生有更高的系统可用性(OR 1.102, 95% CI 1.012-1.200; P=.03)和流程满意度(OR 1.141, 95% CI 1.044-1.247; P=.003)。对在线CME收益的认识受到多种因素的影响。较高的临床经验与临床用药认知呈正相关(OR 1.106, 95% CI 1.004-1.218; P= 0.04),与年龄呈负相关(OR 0.894, 95% CI 0.802-0.996; P= 0.04)。对于与研究相关的获益,阳性预测因子包括与内科相比,妇产科的学科类别(OR 6.217, 95% CI 1.236-31.258; P= 0.03)和初级职称(OR 3.791, 95% CI 1.231-11.673; P= 0.02),而与内科相比,重症监护病房是一个阴性预测因子(OR 0.111, 95% CI 0.014-0.893; P= 0.04)。结论:在线移动CME平台在中国的医疗专业人员中得到了广泛采用,特别是在COVID-19爆发后。然而,课程可用性和用户体验方面的学科差异仍然存在,这表明需要进一步优化课程设计和软件交互。
{"title":"The Design and Evaluation of an Online Continuing Medical Education App for Medical Professionals in China: Quantitative Study.","authors":"Xu Zhang, Xianying He, Yuntian Chu, Dongqing Liu, Minzhao Lyu, Weiyi Wang, Haotian Chen, Meihao Ji, Fangfang Cui, Jie Zhao","doi":"10.2196/76299","DOIUrl":"10.2196/76299","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;As an emerging delivery mode of education, online continuing medical education (CME) increases the accessibility of high-quality medical training for professionals and students in China. Guoyuan (meaning \"nationwide\" in Chinese) is an online CME platform delivered via a mobile app and operated by the National Telemedicine Center of China since 2018, serving as an illustrative case of mobile online CME implementation.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;We identified trends in the adoption and usage of the Guoyuan mobile online CME platform from 2018 to 2023 and provided evidence for the application and optimization of online CME.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We analyzed yearly usage data of the Guoyuan mobile app (The First Affiliated Hospital of Zhengzhou University) in 2018-2023 and collected surveys on the satisfaction and recognition of competency enhancement in online CME in each connected hospital in 2023. Using the IBM SPSS, the nonparametric Kruskal-Wallis H test was used to compare attendance across different disciplines, followed by post hoc pairwise comparisons for course types with significant differences and ordinal logistic regression analysis to examine factors influencing satisfaction with the online CME system and perceived competency enhancement among invited doctors.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;From 2018 to 2023, Guoyuan had 94,537 registered trainees, 1672 published course videos, and 1,878,437 attendances. Attendance was higher for courses in ophthalmology, otolaryngology, and pathology than in other disciplines (median attendance 610, IQR 105-2055 vs 283, IQR 106-690 participants). Based on a sample size of 245 participants, ordinal regression analysis showed that discipline category, professional title, and working years significantly influenced satisfaction. General practitioners showed lower overall satisfaction than internal medicine doctors (odds ratio [OR] 0.323, 95% CI 0.110-0.948; OR 0.251, 95% CI 0.087-0.729; and OR 0.196, 95% CI 0.066-0.585; P=.04; P=.01; P=.003). Junior titles reported higher audio-visual clarity (OR 3.151, 95% CI 1.178-8.427; P=.02) and process satisfaction (OR 4.939, 95% CI 1.674-14.576; P=.004). More experienced doctors had higher system usability (OR 1.102, 95% CI 1.012-1.200; P=.03) and process satisfaction (OR 1.141, 95% CI 1.044-1.247; P=.003). Recognition of online CME's benefits was influenced by multiple factors. Greater clinical experience positively predicted recognition of clinical use (OR 1.106, 95% CI 1.004-1.218; P=.04), while an inverse association was observed with age (OR 0.894, 95% CI 0.802-0.996; P=.04). For research-related benefits, positive predictors included discipline category in obstetrics and gynecology compared to internal medicine (OR 6.217, 95% CI 1.236-31.258; P=.03) and junior professional title (OR 3.791, 95% CI 1.231-11.673; P=.02), whereas intensive care unit was a negative predictor compared to internal medicine (OR 0.111, ","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e76299"},"PeriodicalIF":3.2,"publicationDate":"2026-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12928682/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147277341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Impact of a Structured Training Program on Medical Student Confidence and Behavior During Their First Radial Arterial Puncture: Comparative Study. 结构化训练计划对医学生第一次桡动脉穿刺时信心和行为的影响:比较研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-18 DOI: 10.2196/78086
Camille Rolland-Debord, Lucien Juret, Mathilde Simon, Abdallah El Mouhajer, Cécile Londner, Capucine Morélot-Panzini, Cécile Chenivesse, Thomas Similowski

Background: Radial artery puncture is a common clinical procedure essential for assessing gas exchange but is frequently perceived as stressful by inexperienced operators, who fear causing pain to their patients. Despite its practical relevance, formal training in this procedure is inconsistently integrated into medical curricula. This study evaluated whether a structured training program-combining theoretical instruction, simulation-based practice, and debriefing-could influence students' procedural confidence and decision-making and patient experience during their first clinical arterial puncture.

Objective: This study aimed to determine whether structured simulation-based training influences medical students' anxiety, confidence, and technical performance and patient experience during their first arterial puncture.

Methods: Third-year medical students who had never performed an arterial puncture were assigned to 1 of 2 groups: a structured training group (group 1) or a control group receiving informal or no specific training (group 2). After performing their first arterial puncture under supervision, students completed a questionnaire assessing apprehension, satisfaction, and confidence. The decision to use local anesthesia, puncture success, and patient-rated pain and apprehension were also recorded. A total of 67 students participated (group 1: n=24, 35.8%; group 2: n=43, 64.2%), with 61 patients included. Statistical comparisons were performed using the Fisher exact and nonparametric Mann-Whitney U tests (α=.05).

Results: Self-reported apprehension and confidence were similar between groups. However, group 1 students were significantly less likely to use local anesthesia compared to group 2 students (7/20, 35% vs 28/36, 77.8%, respectively; P=.003), suggesting greater procedural confidence. First-attempt success rates were comparable (group 1: 3/13, 23.1%; group 2: 14/29, 48.3%; P=.18). Median patient-reported pain scores were numerically but not statistically significantly lower when anesthesia was used (2.1, IQR 1.2-4.0 vs 4.8, IQR 2.1-6.4; P=.08).

Conclusions: Structured training influenced students' behavior during their first arterial puncture, reducing reliance on anesthesia despite similar levels of self-reported apprehension. Although confidence ratings did not differ, behavioral indicators suggested improved self-efficacy and readiness for clinical performance. These findings support the behavioral impact of structured procedural education and call for future research using validated assessment tools and long-term follow-up.

背景:桡动脉穿刺是评估气体交换的常见临床操作,但经常被缺乏经验的操作人员认为是有压力的,他们害怕给病人造成疼痛。尽管这一程序具有实际意义,但在医学课程中不一贯地纳入这一程序的正式培训。本研究评估了一个结构化的训练计划——结合理论指导、模拟实践和汇报——是否能影响学生在第一次临床动脉穿刺时的程序信心、决策和患者体验。目的:本研究旨在确定结构化模拟训练是否影响医学生在第一次动脉穿刺时的焦虑、信心、技术表现和患者体验。方法:将从未进行过动脉穿刺的三年级医学生分为两组中的一组:结构化训练组(1组)和对照组(2组),对照组接受非正式或无特殊训练。在监督下进行第一次动脉穿刺后,学生们完成了一份评估恐惧、满意度和信心的问卷。同时记录局部麻醉的决定、穿刺成功率以及患者的疼痛和忧虑程度。共67名学生参与研究(第一组n=24,占35.8%;第二组n=43,占64.2%),共纳入61例患者。采用Fisher精确检验和非参数Mann-Whitney U检验进行统计学比较(α= 0.05)。结果:两组间自我报告的忧虑和信心相似。然而,与2组学生相比,1组学生使用局麻的可能性明显降低(7/ 20,35% vs 28/ 36,77.8%; P= 0.003),表明更大的程序置信度。首次尝试成功率具有可比性(1组:3/13,23.1%;2组:14/29,48.3%;P= 0.18)。使用麻醉时,患者报告的疼痛评分中位数在数值上较低,但在统计学上无显著差异(2.1,IQR 1.2-4.0 vs 4.8, IQR 2.1-6.4; P= 0.08)。结论:结构化训练影响了学生在第一次动脉穿刺时的行为,减少了对麻醉的依赖,尽管自我报告的恐惧水平相似。虽然信心评级没有差异,但行为指标表明自我效能和临床表现的准备程度有所提高。这些发现支持结构化程序性教育的行为影响,并呼吁未来使用有效的评估工具和长期随访进行研究。
{"title":"Impact of a Structured Training Program on Medical Student Confidence and Behavior During Their First Radial Arterial Puncture: Comparative Study.","authors":"Camille Rolland-Debord, Lucien Juret, Mathilde Simon, Abdallah El Mouhajer, Cécile Londner, Capucine Morélot-Panzini, Cécile Chenivesse, Thomas Similowski","doi":"10.2196/78086","DOIUrl":"10.2196/78086","url":null,"abstract":"<p><strong>Background: </strong>Radial artery puncture is a common clinical procedure essential for assessing gas exchange but is frequently perceived as stressful by inexperienced operators, who fear causing pain to their patients. Despite its practical relevance, formal training in this procedure is inconsistently integrated into medical curricula. This study evaluated whether a structured training program-combining theoretical instruction, simulation-based practice, and debriefing-could influence students' procedural confidence and decision-making and patient experience during their first clinical arterial puncture.</p><p><strong>Objective: </strong>This study aimed to determine whether structured simulation-based training influences medical students' anxiety, confidence, and technical performance and patient experience during their first arterial puncture.</p><p><strong>Methods: </strong>Third-year medical students who had never performed an arterial puncture were assigned to 1 of 2 groups: a structured training group (group 1) or a control group receiving informal or no specific training (group 2). After performing their first arterial puncture under supervision, students completed a questionnaire assessing apprehension, satisfaction, and confidence. The decision to use local anesthesia, puncture success, and patient-rated pain and apprehension were also recorded. A total of 67 students participated (group 1: n=24, 35.8%; group 2: n=43, 64.2%), with 61 patients included. Statistical comparisons were performed using the Fisher exact and nonparametric Mann-Whitney U tests (α=.05).</p><p><strong>Results: </strong>Self-reported apprehension and confidence were similar between groups. However, group 1 students were significantly less likely to use local anesthesia compared to group 2 students (7/20, 35% vs 28/36, 77.8%, respectively; P=.003), suggesting greater procedural confidence. First-attempt success rates were comparable (group 1: 3/13, 23.1%; group 2: 14/29, 48.3%; P=.18). Median patient-reported pain scores were numerically but not statistically significantly lower when anesthesia was used (2.1, IQR 1.2-4.0 vs 4.8, IQR 2.1-6.4; P=.08).</p><p><strong>Conclusions: </strong>Structured training influenced students' behavior during their first arterial puncture, reducing reliance on anesthesia despite similar levels of self-reported apprehension. Although confidence ratings did not differ, behavioral indicators suggested improved self-efficacy and readiness for clinical performance. These findings support the behavioral impact of structured procedural education and call for future research using validated assessment tools and long-term follow-up.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e78086"},"PeriodicalIF":3.2,"publicationDate":"2026-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12916088/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146221081","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Artificial Intelligence in Medical Education: Transformative Potential, Current Applications, and Future Implications. 医学教育中的人工智能:变革潜力、当前应用和未来影响。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-17 DOI: 10.2196/77127
Juan S Izquierdo-Condoy, Marlon Arias-Intriago, Laura Montero Corrales, Esteban Ortiz-Prado

Unlabelled: Artificial intelligence (AI) is increasingly influencing medical education by enabling adaptive learning, AI-assisted assessment, and scalable instructional tools. Natural language processing, machine learning, and generative large language models offer innovative ways to support teaching and learning, yet their integration raises ethical, pedagogical, and infrastructural challenges. This viewpoint article aims to examine the current applications, benefits, and challenges of AI in medical education and propose strategies for responsible and effective integration. AI tools such as chatbots, virtual patients, and intelligent tutoring systems enhance personalized and immersive learning. Automated grading and predictive analytics support efficient evaluations, while AI-assisted writing tools streamline content creation. Despite these advances, concerns persist around data privacy, algorithmic bias, unequal access, and diminished critical thinking. Key solutions include AI literacy training, data oversight, equitable infrastructure, and curriculum reform. The FACETS framework offers 6 dimensions (ie, form, application, context, instructional mode, technology, and the SAMR [substitution, augmentation, modification, redefinition model]) to evaluate AI integration effectively. AI offers substantial opportunities to transform medical education, but its adoption must be ethical, equitable, and pedagogically grounded. Strategic frameworks such as FACETS, combined with institutional governance and cross-sector collaboration, are essential to guide implementation so that AI enhances learning outcomes while preserving the humanistic foundations of medical practice.

未标记:人工智能(AI)通过实现自适应学习、人工智能辅助评估和可扩展的教学工具,正日益影响医学教育。自然语言处理、机器学习和生成式大型语言模型为支持教学和学习提供了创新的方法,但它们的整合引发了伦理、教学和基础设施方面的挑战。这篇观点文章旨在研究人工智能在医学教育中的当前应用、好处和挑战,并提出负责任和有效整合的策略。聊天机器人、虚拟病人和智能辅导系统等人工智能工具增强了个性化和沉浸式学习。自动评分和预测分析支持有效的评估,而人工智能辅助写作工具简化了内容创建。尽管取得了这些进步,但对数据隐私、算法偏见、不平等访问和批判性思维减弱的担忧仍然存在。关键的解决方案包括人工智能素养培训、数据监督、公平的基础设施和课程改革。FACETS框架提供了6个维度(即形式、应用、上下文、教学模式、技术和SAMR[替代、增强、修改、重新定义模型])来有效地评估人工智能集成。人工智能为改变医学教育提供了大量机会,但它的采用必须符合道德、公平和教学基础。facet等战略框架与机构治理和跨部门协作相结合,对于指导实施工作至关重要,以便人工智能在提高学习成果的同时保持医疗实践的人文基础。
{"title":"Artificial Intelligence in Medical Education: Transformative Potential, Current Applications, and Future Implications.","authors":"Juan S Izquierdo-Condoy, Marlon Arias-Intriago, Laura Montero Corrales, Esteban Ortiz-Prado","doi":"10.2196/77127","DOIUrl":"10.2196/77127","url":null,"abstract":"<p><strong>Unlabelled: </strong>Artificial intelligence (AI) is increasingly influencing medical education by enabling adaptive learning, AI-assisted assessment, and scalable instructional tools. Natural language processing, machine learning, and generative large language models offer innovative ways to support teaching and learning, yet their integration raises ethical, pedagogical, and infrastructural challenges. This viewpoint article aims to examine the current applications, benefits, and challenges of AI in medical education and propose strategies for responsible and effective integration. AI tools such as chatbots, virtual patients, and intelligent tutoring systems enhance personalized and immersive learning. Automated grading and predictive analytics support efficient evaluations, while AI-assisted writing tools streamline content creation. Despite these advances, concerns persist around data privacy, algorithmic bias, unequal access, and diminished critical thinking. Key solutions include AI literacy training, data oversight, equitable infrastructure, and curriculum reform. The FACETS framework offers 6 dimensions (ie, form, application, context, instructional mode, technology, and the SAMR [substitution, augmentation, modification, redefinition model]) to evaluate AI integration effectively. AI offers substantial opportunities to transform medical education, but its adoption must be ethical, equitable, and pedagogically grounded. Strategic frameworks such as FACETS, combined with institutional governance and cross-sector collaboration, are essential to guide implementation so that AI enhances learning outcomes while preserving the humanistic foundations of medical practice.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e77127"},"PeriodicalIF":3.2,"publicationDate":"2026-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12912660/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146214420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Trust Analysis Canvas for Teaching in the Field of Digital Public Health and Medicine: Tutorial. 数字公共卫生和医学领域教学的信任分析画布:教程。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-17 DOI: 10.2196/79709
Federica Zavattaro, Clara-Maria Barth, Caroline Brall, Viktor von Wyl, Felix Gille

Unlabelled: Trust is increasingly recognized as a cornerstone for the successful implementation of digital public health initiatives, from mobile apps to the use of artificial intelligence in medicine, yet it remains underrepresented in educational curricula. In the course of our research and teaching activities in the field of trust in digital public health and medicine, we identified a gap in existing educational resources that aimed at supporting students in conducting structured trust analyses. Digitalization introduces new complexities into trust relationships, as interactions become increasingly mediated by digital tools. Preparing future professionals, therefore, demands fostering a critical understanding of how trust operates within digital systems, especially in the health sector. To address this gap, we developed and tested the first Trust Analysis Canvas for Teaching (TACT), a tool designed to guide students in conducting trust analyses of case studies in digital public health and medicine. Grounded in conceptual research on trust in health systems and health data sharing, we (1) developed the canvas content and reviewed it with two trust researchers; (2) tested and iteratively refined the tool with 23 students (3 BSc, 14 MSc, and 6 PhD) from diverse disciplines and academic levels through in-person and online focus groups at the universities of Zurich and Bern; (3) collaborated with a graphic designer to optimize its visual layout; and (4) translated the final canvas into French, Italian, German, and Spanish to ensure accessibility across disciplines, academic levels, and languages while maintaining a clear and engaging visual design. This paper introduces TACT, a canvas comprising 16 guiding questions organized around 6 core dimensions, designed to enable students from diverse disciplinary backgrounds and academic levels to engage with the complex concept of trust in a structured and guided manner, thereby addressing the identified gap in the current curricula. We outline the development process and provide a practical, step-by-step tutorial demonstrating its application through a written trust analysis of a digital health case study, supported by references to relevant literature.

未标记:信任日益被认为是成功实施数字公共卫生举措的基石,从移动应用程序到在医学中使用人工智能,但它在教育课程中的代表性仍然不足。在我们在数字公共卫生和医学信任领域的研究和教学活动中,我们发现了现有教育资源的空白,旨在支持学生进行结构化信任分析。数字化给信任关系带来了新的复杂性,因为互动越来越多地由数字工具介导。因此,培养未来的专业人员需要培养对信任如何在数字系统中运作的批判性理解,特别是在卫生部门。为了解决这一差距,我们开发并测试了第一个教学信任分析画布(TACT),该工具旨在指导学生对数字公共卫生和医学案例研究进行信任分析。基于对卫生系统信任和卫生数据共享的概念研究,我们(1)开发了画布内容,并与两位信任研究人员进行了审查;(2)通过苏黎世大学和伯尔尼大学的面对面和在线焦点小组,与来自不同学科和学术水平的23名学生(3名理学士、14名硕士和6名博士)一起测试并反复完善该工具;(3)与平面设计师合作,优化视觉布局;(4)将最终的画布翻译成法语、意大利语、德语和西班牙语,以确保跨学科、学术水平和语言的可访问性,同时保持清晰和引人入胜的视觉设计。本文介绍了TACT,这是一个包含16个指导性问题的画布,围绕6个核心维度组织,旨在使来自不同学科背景和学术水平的学生能够以结构化和引导的方式参与复杂的信任概念,从而解决当前课程中已确定的差距。我们概述了开发过程,并通过对数字健康案例研究的书面信任分析提供了一个实用的、循序渐进的教程,并参考了相关文献。
{"title":"Trust Analysis Canvas for Teaching in the Field of Digital Public Health and Medicine: Tutorial.","authors":"Federica Zavattaro, Clara-Maria Barth, Caroline Brall, Viktor von Wyl, Felix Gille","doi":"10.2196/79709","DOIUrl":"10.2196/79709","url":null,"abstract":"<p><strong>Unlabelled: </strong>Trust is increasingly recognized as a cornerstone for the successful implementation of digital public health initiatives, from mobile apps to the use of artificial intelligence in medicine, yet it remains underrepresented in educational curricula. In the course of our research and teaching activities in the field of trust in digital public health and medicine, we identified a gap in existing educational resources that aimed at supporting students in conducting structured trust analyses. Digitalization introduces new complexities into trust relationships, as interactions become increasingly mediated by digital tools. Preparing future professionals, therefore, demands fostering a critical understanding of how trust operates within digital systems, especially in the health sector. To address this gap, we developed and tested the first Trust Analysis Canvas for Teaching (TACT), a tool designed to guide students in conducting trust analyses of case studies in digital public health and medicine. Grounded in conceptual research on trust in health systems and health data sharing, we (1) developed the canvas content and reviewed it with two trust researchers; (2) tested and iteratively refined the tool with 23 students (3 BSc, 14 MSc, and 6 PhD) from diverse disciplines and academic levels through in-person and online focus groups at the universities of Zurich and Bern; (3) collaborated with a graphic designer to optimize its visual layout; and (4) translated the final canvas into French, Italian, German, and Spanish to ensure accessibility across disciplines, academic levels, and languages while maintaining a clear and engaging visual design. This paper introduces TACT, a canvas comprising 16 guiding questions organized around 6 core dimensions, designed to enable students from diverse disciplinary backgrounds and academic levels to engage with the complex concept of trust in a structured and guided manner, thereby addressing the identified gap in the current curricula. We outline the development process and provide a practical, step-by-step tutorial demonstrating its application through a written trust analysis of a digital health case study, supported by references to relevant literature.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e79709"},"PeriodicalIF":3.2,"publicationDate":"2026-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12912458/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146214431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
AI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI-Simulated Patient System: Cross-Sectional Validation Study. 生成式人工智能模拟患者系统中医疗访谈记录的人工智能与人类评估:横断面验证研究。
IF 3.2 Q1 EDUCATION, SCIENTIFIC DISCIPLINES Pub Date : 2026-02-17 DOI: 10.2196/81673
Hiromizu Takahashi, Kiyoshi Shikino, Takeshi Kondo, Yuji Yamada, Yoshitaka Tomoda, Minoru Kishi, Yuki Aiyama, Sho Nagai, Akiko Enomoto, Yoshinori Tokushima, Takahiro Shinohara, Fumiaki Sano, Takeshi Matsuura, Rikiya Watanabe, Toshio Naito
<p><strong>Background: </strong>Generative artificial intelligence (AI) is increasingly used in medical education, including AI-based virtual patients to improve interview skills. However, how much AI-based assessment (ABA) differs from human-based assessment (HBA) remains unclear.</p><p><strong>Objective: </strong>This study aimed to compare the quality of clinical interview assessments generated via an ABA (GPT-o1 Pro [ABA-o1] and GPT-5 Pro [ABA-5]) with those generated via an HBA conducted by clinical instructors in an AI-based virtual patient setting. We also examined whether AI reduced evaluation time and assessed agreement across participants with different levels of clinical experience.</p><p><strong>Methods: </strong>A standardized case of leg weakness was implemented in an AI-based virtual patient. Seven participants (2 medical students, 3 residents, and 2 attending physicians) each conducted an interview with the AI patient, and transcripts were scored using the 25-item Master Interview Rating Scale (0-125). Three evaluation strategies were compared. First, GPT-o1 Pro and GPT-5 Pro scored each transcript 5 times with different random seeds to test case specificity. Processing time was logged automatically. Second, 5 blinded clinical instructors independently rated each transcript once using the same rubric. Third, reliability metrics were applied. For AI, intraclass correlation coefficients (ICCs) quantified repeatability. For humans, the ICC(2,1) was calculated. Agreement was quantified using the Pearson r, Lin concordance correlation coefficient, Bland-Altman limits of agreement, Cronbach α, and ICC. Time efficiency was expressed as mean minutes per transcript and relative percentage reduction.</p><p><strong>Results: </strong>Mean interview scores were similar across methods (ABA-o1: mean 52.1, SD 6.9; ABA-5: mean 53.2, SD 6.8; HBA: mean 53.7, SD 6.8). Agreement between ABA and HBA was strong (r=0.90; concordance correlation coefficient=0.88) with minimal bias (ABA-o1: mean 0.4, SD 2.7; ABA-5: mean 1.5, SD 5.2; limits of agreement: -4.9 to 5.7 for ABA-o1 and -8.6 to 11.7 for ABA-5). The Cronbach α was 0.81 (ABA-o1), 0.86 (ABA-5), and 0.80 (HBA); the ICC(3,1) was 0.77 (ABA-o1) and 0.82 (ABA-5); and the ICC(2,1) was 0.38 (HBA). The coefficient of variation for ABA was approximately half that of HBA (6.6% vs 13.9%). Processing time for 5 runs was 4 minutes, 19 seconds for ABA-o1 and 3 minutes, 20 seconds for ABA-5 vs 10 minutes, 16 seconds for physicians, corresponding to 58% and 67.6% reductions, respectively.</p><p><strong>Conclusions: </strong>ABA-o1 and ABA-5 produced scores closely matching HBA while demonstrating superior consistency and reliability. In the setting of virtual interview transcripts, these findings suggest that ABA may serve as a valid, rapid, and scalable alternative to HBA, reducing per-assessment time by over half. Applied strategically, AI-based scoring could enable timely feedback, improve efficiency, and reduce
背景:生成式人工智能(AI)越来越多地应用于医学教育,包括基于AI的虚拟患者来提高面试技巧。然而,基于人工智能的评估(ABA)与基于人类的评估(HBA)有多大不同尚不清楚。目的:本研究旨在比较通过ABA (gpt - 01 Pro [ABA- 01]和GPT-5 Pro [ABA-5])生成的临床访谈评估的质量与临床教师在基于人工智能的虚拟患者环境中通过HBA生成的临床访谈评估的质量。我们还研究了人工智能是否缩短了评估时间,并评估了具有不同临床经验水平的参与者的一致性。方法:在一个基于人工智能的虚拟患者中实施一个标准化的腿部无力病例。7名参与者(2名医学生、3名住院医生和2名主治医生)分别对AI患者进行了访谈,并使用25项主访谈量表(0-125)对成绩单进行评分。比较了三种评价策略。首先,gpt - 01 Pro和GPT-5 Pro用不同的随机种子对每个转录本进行5次评分,以测试病例特异性。自动记录处理时间。其次,5名盲法临床讲师使用相同的评分标准对每份成绩单进行独立评分。第三,应用可靠性指标。对于人工智能,类内相关系数(ICCs)量化了重复性。对于人类,计算了ICC(2,1)。使用Pearson r、Lin一致性相关系数、Bland-Altman一致性极限、Cronbach α和ICC对一致性进行量化。时间效率表示为每个转录本的平均分钟数和相对减少百分比。结果:不同方法的平均访谈得分相似(aba - 01:平均52.1,SD 6.9; ABA-5:平均53.2,SD 6.8; HBA:平均53.7,SD 6.8)。ABA和HBA之间的一致性很强(r=0.90;一致性相关系数=0.88),偏差最小(ABA- 01:平均0.4,SD 2.7; ABA-5:平均1.5,SD 5.2; ABA- 01的一致性限为-4.9至5.7,ABA-5的一致性限为-8.6至11.7)。Cronbach α分别为0.81 (aba - 01)、0.86 (ABA-5)和0.80 (HBA);ICC(3,1)分别为0.77 (aba - 01)和0.82 (ABA-5);ICC(2,1)为0.38 (HBA)。ABA的变异系数大约是HBA的一半(6.6% vs 13.9%)。5组处理时间aba - 01组为4分19秒,ABA-5组为3分20秒,医生组为10分16秒,分别减少58%和67.6%。结论:aba - 01和ABA-5的评分与HBA非常匹配,且具有较好的一致性和可靠性。在虚拟访谈记录的设置中,这些发现表明ABA可以作为HBA的有效、快速和可扩展的替代方案,将每次评估时间减少一半以上。策略性地应用,基于人工智能的评分可以实现及时反馈,提高效率,减少教师的工作量。需要进一步的研究来证实在更广泛的情况下的普遍性。
{"title":"AI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI-Simulated Patient System: Cross-Sectional Validation Study.","authors":"Hiromizu Takahashi, Kiyoshi Shikino, Takeshi Kondo, Yuji Yamada, Yoshitaka Tomoda, Minoru Kishi, Yuki Aiyama, Sho Nagai, Akiko Enomoto, Yoshinori Tokushima, Takahiro Shinohara, Fumiaki Sano, Takeshi Matsuura, Rikiya Watanabe, Toshio Naito","doi":"10.2196/81673","DOIUrl":"10.2196/81673","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Generative artificial intelligence (AI) is increasingly used in medical education, including AI-based virtual patients to improve interview skills. However, how much AI-based assessment (ABA) differs from human-based assessment (HBA) remains unclear.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to compare the quality of clinical interview assessments generated via an ABA (GPT-o1 Pro [ABA-o1] and GPT-5 Pro [ABA-5]) with those generated via an HBA conducted by clinical instructors in an AI-based virtual patient setting. We also examined whether AI reduced evaluation time and assessed agreement across participants with different levels of clinical experience.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;A standardized case of leg weakness was implemented in an AI-based virtual patient. Seven participants (2 medical students, 3 residents, and 2 attending physicians) each conducted an interview with the AI patient, and transcripts were scored using the 25-item Master Interview Rating Scale (0-125). Three evaluation strategies were compared. First, GPT-o1 Pro and GPT-5 Pro scored each transcript 5 times with different random seeds to test case specificity. Processing time was logged automatically. Second, 5 blinded clinical instructors independently rated each transcript once using the same rubric. Third, reliability metrics were applied. For AI, intraclass correlation coefficients (ICCs) quantified repeatability. For humans, the ICC(2,1) was calculated. Agreement was quantified using the Pearson r, Lin concordance correlation coefficient, Bland-Altman limits of agreement, Cronbach α, and ICC. Time efficiency was expressed as mean minutes per transcript and relative percentage reduction.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Mean interview scores were similar across methods (ABA-o1: mean 52.1, SD 6.9; ABA-5: mean 53.2, SD 6.8; HBA: mean 53.7, SD 6.8). Agreement between ABA and HBA was strong (r=0.90; concordance correlation coefficient=0.88) with minimal bias (ABA-o1: mean 0.4, SD 2.7; ABA-5: mean 1.5, SD 5.2; limits of agreement: -4.9 to 5.7 for ABA-o1 and -8.6 to 11.7 for ABA-5). The Cronbach α was 0.81 (ABA-o1), 0.86 (ABA-5), and 0.80 (HBA); the ICC(3,1) was 0.77 (ABA-o1) and 0.82 (ABA-5); and the ICC(2,1) was 0.38 (HBA). The coefficient of variation for ABA was approximately half that of HBA (6.6% vs 13.9%). Processing time for 5 runs was 4 minutes, 19 seconds for ABA-o1 and 3 minutes, 20 seconds for ABA-5 vs 10 minutes, 16 seconds for physicians, corresponding to 58% and 67.6% reductions, respectively.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;ABA-o1 and ABA-5 produced scores closely matching HBA while demonstrating superior consistency and reliability. In the setting of virtual interview transcripts, these findings suggest that ABA may serve as a valid, rapid, and scalable alternative to HBA, reducing per-assessment time by over half. Applied strategically, AI-based scoring could enable timely feedback, improve efficiency, and reduce","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"12 ","pages":"e81673"},"PeriodicalIF":3.2,"publicationDate":"2026-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12912650/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146214342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
JMIR Medical Education
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1