JMIR Medical Education最新文献

Background: Team performance is crucial in crisis situations. Although the Thai version of Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) has been validated, challenges remain due to its subjective evaluation. To date, no studies have examined the relationship between electroencephalogram (EEG) activity and team performance, as assessed by TeamSTEPPS, during virtual simulation-based interprofessional education (SIMBIE), where face-to-face communication is absent.

Objective: This study aims to investigate the correlation between EEG-based brain-to-brain synchronization and TeamSTEPPS scores in multiprofessional teams participating in virtual SIMBIE sessions.

Methods: This single-center study involved 90 participants (15 groups of 6 simulated professionals: 1 medical doctor, 2 nurses, 1 pharmacist, 1 medical technologist, and 1 radiological technologist). Each group completed two 30-minute virtual SIMBIE sessions focusing on team training in a crisis situation involving COVID-19 pneumonia with a difficult airway, resulting in 30 sessions in total. The TeamSTEPPS scores of each participant across 5 domains were independently assessed by 2 trained raters based on screen recording, and their average values were used. The scores of participants in the same session were aggregated to generate a group TeamSTEPPS score, representing group-level performance. EEG data were recorded using wireless EEG acquisition devices and computed for total interdependence (TI), which represents brain-to-brain synchronization. The TI values of participants in the same session were aggregated to produce a group TI, representing group-level brain-to-brain synchronization. We investigated the Pearson correlations between the TI and the scores at both the group and individual levels.

Results: Interrater reliability for the TeamSTEPPS scores among 12 raters indicated good agreement on average (mean 0.73, SD 0.18; range 0.32-0.999). At the individual level, the Pearson correlations between the TI and the scores were weak and not statistically significant across all TeamSTEPPS domains (all adjusted P≥.05). However, strongly negative, statistically significant correlations between the group TI and the group TeamSTEPPS scores in the alpha frequency band (8-12 Hz) of the anterior brain area were found across all TeamSTEPPS domains after correcting for multiple comparisons (mean -0.87, SD 0.06; range -0.93 to -0.8).

Conclusions: Strong negative correlations between the group TI and the group TeamSTEPPS scores were observed in the anterior alpha activity during online hexad virtual SIMBIE. These findings suggest that anterior alpha TI may serve as an objective metric for assessing TeamSTEPPS-based team performance.

Background: Improving the quality of education in clinical settings requires an understanding of learners' experiences and learning processes. However, this is a significant burden on learners and educators. If learners' learning records could be automatically analyzed and their experiences could be visualized, this would enable real-time tracking of their progress. Large language models (LLMs) may be useful for this purpose, although their accuracy has not been sufficiently studied.

Objective: This study aimed to explore the accuracy of predicting the actual clinical experiences of medical students from their learning log data during clinical clerkship using LLMs.

Methods: This study was conducted at the Nagoya University School of Medicine. Learning log data from medical students participating in a clinical clerkship from April 22, 2024, to May 24, 2024, were used. The Model Core Curriculum for Medical Education was used as a template to extract experiences. OpenAI's ChatGPT was selected for this task after a comparison with other LLMs. Prompts were created using the learning log data and provided to ChatGPT to extract experiences, which were then listed. A web application using GPT-4-turbo was developed to automate this process. The accuracy of the extracted experiences was evaluated by comparing them with the corrected lists provided by the students.

Results: A total of 20 sixth-year medical students participated in this study, resulting in 40 datasets. The overall Jaccard index was 0.59 (95% CI 0.46-0.71), and the Cohen κ was 0.65 (95% CI 0.53-0.76). Overall sensitivity was 62.39% (95% CI 49.96%-74.81%), and specificity was 99.34% (95% CI 98.77%-99.92%). Category-specific performance varied: symptoms showed a sensitivity of 45.43% (95% CI 25.12%-65.75%) and specificity of 98.75% (95% CI 97.31%-100%), examinations showed a sensitivity of 46.76% (95% CI 25.67%-67.86%) and specificity of 98.84% (95% CI 97.81%-99.87%), and procedures achieved a sensitivity of 56.36% (95% CI 37.64%-75.08%) and specificity of 98.92% (95% CI 96.67%-100%). The results suggest that GPT-4-turbo accurately identified many of the actual experiences but missed some because of insufficient detail or a lack of student records.

Conclusions: This study demonstrated that LLMs such as GPT-4-turbo can predict clinical experiences from learning logs with high specificity but moderate sensitivity. Future improvements in AI models, providing feedback to medical students' learning logs and combining them with other data sources such as electronic medical records, may enhance the accuracy. Using artificial intelligence to analyze learning logs for assessment could reduce the burden on learners and educators while improving the quality of educational assessments in medical education.

Unlabelled: The release of GPT-4 Omni (GPT-4o), an advanced multimodal generative artificial intelligence (AI) model, generated substantial enthusiasm in the field of higher education. However, one year later, medical education continues to face significant challenges, demonstrating the need to move from initial experimentation with the integration of multimodal AIs in medical education toward meaningful integration. In this Viewpoint, we argue that GPT-4o's true value lies not in novelty, but in its potential to enhance training in communication skills, clinical reasoning, and procedural skills by offering real-time simulations and adaptive learning experiences using text, audio, and visual inputs in a safe, immersive, and cost-effective environment. We explore how this innovation has made it possible to address key medical educational challenges by simulating realistic patient interactions, offering personalized feedback, and reducing educator workloads and costs, where traditional teaching methods struggle to replicate the complexity and dynamism of real-world clinical scenarios. However, we also address the critical challenges of this approach, including data accuracy, bias, and ethical decision-making. Rather than seeing GPT-4o as a replacement, we propose its use as a strategic supplement, scaffolded into curriculum frameworks and evaluated through ongoing research. As the focus shifts from AI novelty to sustainable implementation, we call on educators, policymakers, and curriculum designers to establish governance mechanisms, pilot evaluation strategies, and develop faculty training. The future of AI in medical education depends not on the next breakthrough, but on how we integrate today's tools with intention and rigor.

Background: The integration of digital technologies is becoming increasingly essential in cancer care. However, limited digital health literacy among clinical and nonclinical cancer health care professionals poses significant challenges to effective implementation and sustainability over time. To address this, the European Union is prioritizing the development of targeted digital skills training programs for cancer care providers, the TRANSiTION project among them. A crucial initial step in this effort is conducting a comprehensive gap analysis to identify specific training needs.

Objective: The aim of this work is to identify training gaps and prioritize the digital skill development needs in the oncology health care workforce.

Methods: An importance-performance analysis (IPA) was conducted following a survey that assessed the performance and importance of 7 digital skills: information, communication, content creation, safety, eHealth problem-solving, ethics, and patient empowerment.

Results: A total of 67 participants from 11 European countries completed the study: 38 clinical professionals (CP), 16 nonclinical professionals (NCP), and 13 patients or caregivers (PC). CP acknowledged the need for a comprehensive training program that includes all 7 digital skills. Digital patient empowerment and safety skills emerge as the highest priorities for both CP and NCP. Conversely, NCP assigned a lower priority to digital content creation skills, and PC assigned a lower priority to digital information and ethical skills. The IPA also revealed discrepancies in digital communication skills across groups (H=6.50; P=.04).

Conclusions: The study showcased the pressing need for comprehensive digital skill training for cancer health care professionals across diverse backgrounds and health care systems in Europe, tailored to their occupation and care setting. Incorporating PC perspectives ensures a balanced approach to addressing these training gaps. These findings provide a valuable knowledge base for designing digital skills training programs, promoting a holistic approach that integrates the perspectives of the various stakeholders involved in digital cancer care.

Background: Research capacity building (RCB) among health care professionals remains limited, particularly for those working outside academic institutions. Japan is experiencing a decline in original clinical research due to insufficient RCB infrastructure. Our previous hospital-based workshops were effective but faced geographical and sustainability constraints. We developed a fully online Scientific Research Works Peer Support Group (SRWS-PSG) model that addresses geographical and time-bound constraints and establishes a sustainable economic model. Mentees use online materials, receive support from mentors via a communication platform after formulating their research question, and transition into mentors upon publication.

Objective: We evaluated whether our model's theoretical benefits translated into actual program effectiveness in RCB among health care professionals.

Methods: We conducted a retrospective cohort study of health care professionals who participated in the SRWS-PSG program between September 2019 and January 2025. Mentees progressed through a structured modular curriculum covering systematic review methodology, from protocol development to manuscript preparation, with personalized mentoring support. We evaluated manuscript submission, program discontinuation, promotion to a mentor status, and mentor response time. We collected data from program records and chat logs. Manuscript submission was defined as mentor-confirmed submission of a systematic review manuscript to a peer-reviewed journal. Program discontinuation referred to formal withdrawal before manuscript submission. Mentor promotion was defined as acceptance of an invitation to serve as a junior mentor after manuscript submission. Mentor response time was the elapsed time from a mentee's question in the chat to the first reply by an assigned mentor.

Results: Of 85 mentees analyzed, 31 (36.5%) held academic degrees (PhD or MPH), and 68 (80%) were medical doctors. During a median follow-up of 10 months, 51 (60%) submitted manuscripts and 46 (90%) became mentors. Ten mentees (12%) discontinued the program. The median mentor response time was 0.8 hours, with 90% responding within 24 hours.

Conclusions: A majority of participants of SRWS-PSG submitted manuscripts. This fully online RCB program might address geographical barriers and provides an adaptable approach for RCB across diverse health care contexts.

Unlabelled: The integration of large language models into medical education has significantly increased, providing valuable assistance in single-turn, isolated educational tasks. However, their utility remains limited in complex, iterative instructional workflows characteristic of clinical education. Single-prompt AI chatbots lack the necessary contextual awareness and iterative capability required for nuanced educational tasks. This Viewpoint paper argues for a shift from conventional chatbot paradigms toward a modular, multistep artificial intelligence (AI) agent framework that aligns closely with the pedagogical needs of medical educators. We propose a modular framework composed of specialized AI agents, each responsible for distinct instructional subtasks. Furthermore, these agents operate within clearly defined boundaries and are equipped with tools and resources to accomplish their tasks and ensure pedagogical continuity and coherence. Specialized agents enhance accuracy by using models optimally tailored to specific cognitive tasks, increasing the quality of outputs compared to single-model workflows. Using a clinical scenario design as an illustrative example, we demonstrate how task specialization, iterative feedback, and tool integration in an agent-based pipeline can mirror expert-driven educational processes. The framework maintains a human-in-the-loop structure, with educators reviewing and refining each output before progression, ensuring pedagogical integrity, flexibility, and transparency. Our proposed shift toward modular AI agents offers significant promise for enhancing educational workflows by delegating routine tasks to specialized systems. We encourage educators to explore how these emerging AI ecosystems could transform medical education.

Background: Large language models such as ChatGPT (OpenAI) have shown promise in medical education assessments, but the comparative effects of prompt engineering across optimized variants and relative performance against medical students remain unclear.

Objective: This study aims to systematically evaluate the impact of prompt engineering on five ChatGPT variants (GPT-3.5, GPT-4.0, GPT-4o, GPT-4o1-mini, and GPT-4o1) and benchmark their performance against fourth-year medical students in midterm and final examinations.

Methods: A 100-item examination dataset covering multiple choice questions, short answer questions, clinical case analysis, and image-based questions was administered to each model under no-prompt and prompt-engineering conditions over 5 independent runs. Student cohort scores (N=143) were collected for comparison. Responses were scored using standardized rubrics, converted to percentages, and analyzed in SPSS Statistics (v29.0) with paired t tests and Cohen d (P<.05).

Results: Baseline midterm scores ranged from 59.2% (GPT-3.5) to 94.1% (GPT-4o1), and final scores ranged from 55% to 92.4%. Fourth-year students averaged 89.4% (midterm) and 80.2% (final). Prompt engineering significantly improved GPT-3.5 (10.6%, P<.001) and GPT-4.0 (3.2%, P=.002) but yielded negligible gains for optimized variants (P=.07-.94). Optimized models matched or exceeded student performance on both exams.

Conclusions: Prompt engineering enhances early-generation model performance, whereas advanced variants inherently achieve near-ceiling accuracy, surpassing medical students. As large language models mature, emphasis should shift from prompt design to model selection, multimodal integration, and critical use of artificial intelligence as a learning companion.

Background: Traditional Chinese medicine (TCM) education in China has evolved significantly, shaped by both national policy and social needs. Despite this, the academic community has yet to fully explore the long-term trends and core issues in TCM education policies. As the global interest in TCM continues to grow, understanding these trends becomes crucial for guiding future policy and educational reforms. This study used cutting-edge deep learning techniques to fill this gap, offering a novel, data-driven perspective on the evolution of TCM education policies.

Objective: This study aimed to systematically analyze the research topics and evolutionary trends in TCM education policies in China using a deep learning-based topic modeling approach, providing valuable insights to guide future policy development and educational practices.

Methods: TCM policy-related documents were collected from major sources, including the Ministry of Education, the National Administration of Traditional Chinese Medicine, PKU Lawinfo, and archives of TCM colleges. The text was preprocessed and analyzed using the BERTopic model, a state-of-the-art tool for topic modeling, to extract key themes and examine the policy development trajectory.

Results: The analysis revealed 27 core topics in TCM education policies, including medical education, curriculum reform, rural health care, internationalization, and the integration of TCM with modern education systems. These topics were clustered into 5 stages of policy evolution: marginalization, standardization, specialization, systematization, and restandardization. These stages reflect the ongoing balancing act between modernizing TCM education and preserving its traditional values, while adapting to national political, social, and economic strategies.

Conclusions: This study offers groundbreaking insights into the dynamic and multifaceted evolution of TCM education policies in China. By leveraging the BERTopic model, it provides a comprehensive framework for understanding the forces shaping TCM education and offers actionable recommendations for future policy making. The findings are essential for educators, policymakers, and researchers aiming to refine and innovate TCM education in an increasingly globalized world.

Background: Training of health care professionals and their participation in continuous medical education are crucial to ensure quality health care. Low-resource countries in Sub-Saharan Africa struggle with health care disparities between urban and rural areas concerning access to educational resources. While e-learning can facilitate a wide distribution of educational content, it depends on learners' engagement and infrastructure.

Objective: This study aims to assess knowledge, attitude, practice, and access to infrastructure related to e-learning among health care professionals in primary health care settings in Ethiopia.

Methods: In April 2023, we carried out a quantitative, questionnaire-based cross-sectional study guided by the knowledge, attitudes, and practice framework, including additional items on available infrastructure. The scores in each category are defined as "high" and "low" based on the median, followed by the application of logistic regression on selected sociodemographic factors. We included health care professionals working in general and primary hospitals, health centers, and health posts.

Results: Of 398 participants (response rate 94.5%), more than half (n=207, 52%) reported feeling confident about their understanding of e-learning and conducting online searches, both for general (n=247, 62.1%) and medical-related content (n=251, 63.1%). Higher levels of education were associated with better knowledge (adjusted odds ratio [AOR] 2.32, 95% CI 1.45-3.68). Regardless of financial and personal efforts, we observed a generally positive attitude. Almost half of the participants (n=172, 43.2%) reported using the internet daily, compared to 16.8% (n=67) of participants who never used the internet. Higher education (AOR 2.56, 95% CI 1.57-4.16) and income levels (AOR 1.31, 95% CI 1.06-1.62) were associated with higher practice scores of e-learning-related activities. Women, however, exhibited lower practice scores (AOR 0.44, 95% CI 0.27-0.71). Regular access to an internet-enabled device was reported by 43.5% (n=173) of the participants. Smartphones were the primarily used device (268/393, 67.3%). Common barriers to internet access were limited internet availability (142/437, 32.5%) and costs (n=190, 43.5%). Higher education (AOR 1.56, 95% CI 0.98, 2.46) and income (AOR 1.50; 95% CI 1.21-1.85) were associated with increased access to infrastructure, while it was decreased for women (AOR 0.48, 95% CI 0.30-0.77).

Conclusions: Although Ethiopian health care professionals report mixed levels of knowledge, they have a positive attitude toward e-learning in medical education. While internet use is common, especially via smartphone, the access to devices and reliable internet is limited. To improve accessibility, investments in the digital infrastructure and individual digital education programs are necessary, especially targetin