JMIR Medical Education最新文献

Background: Digital tools are increasingly used to support clinical decision-making in dental education. However, the accuracy and efficiency of different support tools, including generative artificial intelligence, in the context of dental trauma management remain underexplored.

Objective: This study aimed to evaluate the accuracy of various information sources (chatbot, textbook, mobile app, and no support tool) in conveying clinically relevant educational content related to decision-making in the primary care of traumatically injured teeth. Additionally, the effect of the input strategy on the chatbot's output response was evaluated.

Methods: Fifty-nine dental students with limited prior experience in dental trauma were randomly assigned to one of 4 groups: chatbot (based on generative pretrained transformer [GPT]-4o, n=15), digital textbook (n=15), mobile app (AcciDent app 3.5, n=15), and control group (no support tool, n=14). Participants answered 25 dichotomous questions in a digital examination format using the information source allocated to their group. The primary outcome measures were the percentage of correct responses and the time required to complete the examination. Additionally, for the group using ChatGPT-4o, the quality of prompts and the clarity of chatbot responses were independently evaluated by 2 calibrated examiners using a 5-point Likert scale. Statistical analyses included nonparametric analyses using Kruskal-Wallis tests and mixed-effects regression analyses with an α level of .05.

Results: All support tools led to a significantly higher accuracy compared with the control group (P<.05), with mean accuracies of 87.47% (SD 5.63%), 86.40% (SD 5.19%), and 86.40% (SD 6.38%) for the textbook, the AcciDent app, and ChatGPT-4o, respectively. The groups using the chatbot and the mobile app required significantly less time than the textbook group (P<.05). Within the ChatGPT-4o group, higher prompt quality was associated with greater clarity of the chatbot's responses (odds ratio 1.44, 95% CI 1.13-1.83, P<.05), which in turn increased the likelihood of students selecting the correct answers (odds ratio 1.89, 95% CI 1.26-2.80, P<.05).

Conclusions: ChatGPT-4o and the AcciDent app can serve dental students as an accurate and time-efficient support tool in dental trauma care. However, the performance of ChatGPT-4o varies with the precision of the input prompt, underscoring the necessity for users to critically evaluate artificial intelligence-generated responses.

Trial registration: OSF Registries 10.17605/OSF.IO/XW62J; https://osf.io/xw62j/overview.

Background: Video-sharing sites such as YouTube (Google) and TikTok (ByteDance) have become indispensable resources for learners and educators. The recent growth in generative artificial intelligence (AI) tools, however, has resulted in low-quality, AI-generated material (commonly called "slop") cluttering these platforms and competing with authoritative educational materials. The extent to which slop has polluted science education video content is unknown, as are the specific hazards to learning from purportedly educational videos made by AI without the use of human discretion.

Objective: This study aimed to advance a formal definition of slop (based on the recent theoretical construct of "careless speech"), to identify its qualitative characteristics that may be problematic for learners, and to gauge its prevalence among preclinical biomedical science (medical biochemistry and cell biology) videos on YouTube and TikTok. We also examined whether any quantitative features of video metadata correlate with the presence of slop.

Methods: An automated search of publicly available YouTube and TikTok videos related to 10 search terms was conducted in February and March 2025. After exclusion of duplicates, off-topic, and non-English results, videos were screened, and those suggestive of AI were flagged. The flagged videos were subject to a 2-stage qualitative content analysis to identify and code problematic features before an assignment of "slop" was made. Quantitative viewership data on all videos in the study were scraped using automated tools and compared between slop videos and the overall population.

Results: We define "slop" according to the degree of human care in production. Of 1082 videos screened (814 YouTube, 268 TikTok), 57 (5.3%) were deemed probably AI-generated and low-quality. From qualitative analysis of these and 6 additional AI-generated videos, we identified 16 codes for problematic aspects of the videos as related to their format or contents. These codes were then mapped to the 7 characteristics of careless speech identified earlier. Analysis of view, like, and comment rates revealed no significant difference between slop videos and the overall population.

Conclusions: We find slop to be not especially prevalent on YouTube and TikTok at this time. These videos have comparable viewership statistics to the overall population, although the small dataset suggests this finding should be interpreted with caution. From the slop videos that were identified, several features inconsistent with best practices in multimedia instruction were defined. Our findings should inform learners seeking to avoid low-quality material on video-sharing sites and suggest pitfalls for instructors to avoid when making high-quality educational materials with generative AI.

Background: Artificial intelligence (AI) is increasingly being integrated into medical education. As AI technologies continue to evolve, they are expected to enable more sophisticated student tutoring, performance evaluation, and reforms of curricula. However, medical education entities have been ill-prepared to embrace this technological revolution, and there is anxiety concerning its potential harm to the community.

Objective: To explore research trends in the field and identify future directions for AI-enabled medical education, we conducted a systematic bibliometric analysis focusing on temporal trajectories in the field.

Methods: Documents were collected from the Web of Science and Scopus databases covering the period from 2000 to 2024. A multistep search strategy combining information retrieval, a definitive journal list, and cocitation analysis was used to identify relevant publications. Journal and author impact were assessed using both publication and citation metrics. Research trends and hot spots were examined through citation burst detection, frequency analysis, and co-occurrence networks, with a color gradient used to indicate the average occurrence year of keywords. The citation lineage structure of the field was evaluated using a k-means clustering-based analysis of cocitation networks to trace influential references.

Results: Our analysis revealed a significant increase in publications since 2021, with foundational works emerging as early as 2019. Influential journals in this domain included JMIR Medical Education, Anatomical Sciences Education, and Medical Education. The evolving research trajectory exhibited a shift from conventional computer-assisted learning tools toward generative AI platforms. Earlier applications of AI in medical education were predominantly concentrated at the undergraduate level, indicating substantial potential for expansion into graduate and continuing medical education. Furthermore, limited cocitation connections were observed between recent generative AI research and conventional medical AI studies, and investigations into medical students' attitudes toward generative AI remain scarce.

Conclusions: There are critical needs for (1) interdisciplinary studies that intentionally integrate generative AI with foundational medical AI work and (2) involving medical educators and students in AI development. Future research should focus on building theoretical frameworks and collaborative projects that connect these currently separate domains to foster a more cohesive knowledge base.

Background: Deepseek-R1, an open-source large language model (LLM), has generated significant global interest in the past months.

Objective: This study aimed to compare the performance of DeepSeek and OpenAI LLMs on the Chinese National Medical Licensing Examination (NMLE) and evaluate their potential in medical education.

Methods: This cross-sectional study assessed 2 DeepSeek models (DeepSeek-R1 and DeepSeek-V3), 3 OpenAI models (ChatGPT-o1 pro, ChatGPT-o3 mini, and GPT-4o), and 2 additional Chinese LLMs (ERNIE 4.5 Turbo and Qwen 3) using the 2021 NMLE. Model performance was evaluated based on overall accuracy, accuracy across question types (A1, A2, A3 and A4, and B1), case analysis and non-case analysis questions, medical specialties, and accuracy consensus between different model combinations.

Results: All LLMs successfully passed the NMLE. DeepSeek-R1 achieved the highest accuracy (573/597, 96%), followed by DeepSeek-V3 (558/600, 93%), both of which significantly outperformed ChatGPT-o1 pro (450/600, 75%), ChatGPT-o3 mini (455/600, 75.8%), and GPT-4o (452/600, 75.3%; P<.001 for all comparisons). Performance disparities were consistent across various question types (A1, A2, A3 and A4, and B1), case analysis and non-case analysis questions, different types of case analyses, and medical specialties. The accuracy consensus between DeepSeek-R1 and DeepSeek-V3 reached 97.7% (544/557), significantly outperforming DeepSeek-R1 alone (P=.04). Two additional Chinese LLMs, ERNIE 4.5 Turbo (572/600, 95.3%) and Qwen 3 (555/600, 92.5%), also exhibited significantly better performance compared to the 3 OpenAI models (all P<.001).

Conclusions: This study demonstrates that DeepSeek-R1 and DeepSeek-V3 significantly outperform OpenAI models on the NMLE. DeepSeek models show promise as tools for medical education and exam preparation in the Chinese language.

Unlabelled: An April 2025 survey of 118 first-year Japanese medical students found high use of generative artificial intelligence (84.7%) but limited formal learning (49.2%), with strong learning interest yet neutral assignment use, indicating a need for structured literacy in generative artificial intelligence.

Background: Project Extension for Community Healthcare Outcomes (ECHO) is an innovative model to increase capacity to treat patients in their community. Despite a growing body of evidence supporting its effectiveness, little is known about the implementation processes of multiple ECHO programs within an institution from the perspective of executives and institutional leaders.

Objective: The study objective was to explore from an institutional and organizational standpoint the systemic characteristics that influence the implementation of Project ECHO programs, their growth within an ecosystem, and their sustainability.

Methods: Focus groups and individual interviews were carried out with executives and leaders from an institution that implemented 3 Project ECHO programs, and verbatim were analyzed based on organizational readiness and implementation tools for Project ECHO.

Results: This study highlighted the rarely reported perspectives of executives and institutional partners, shedding light on the organizational components that are essential to the deployment and sustainability of Project ECHO. Results reflect the intricate balance between institutional resources and its broader mission within a provincial, public health care system. In terms of acceptability, the fit between the projects and the institution's values of innovation, contribution to the broader community, and improving patient trajectory was central from the organizational leaders' standpoint. The structure of the projects and their rapid growth within the institution confirmed the adequacy with the institution. The projects benefited from temporary funds initially, and the lack of performance indicators that were easily measurable and the lack of recognition for invested time from clinicians were barriers to moving toward sustainability. Organizational characteristics, including a decentralized management structure and ministerial support for innovative educational practices, increased the perceived feasibility of implementing and maintaining these programs.

Conclusions: This qualitative study of institution leaders and directors highlighted the challenges and facilitators to the deployment of an innovative continuous education model aimed at building capacity in the community for the management of various health conditions. Despite limitations, such as temporary initial funding, challenges in collecting performance indicators, most valued, and rigidity of the projects' structure, results also show many characteristics (innovative model, alignment with the institution's mission, and simplicity of its deployment) that helped move these projects toward sustainability within the institution. Results offer learning experiences that will be relevant to other settings evolving within a similar public health care system, wanting to implement this model.

Background: Free Open Access Medical Education has the potential to democratize access to medical knowledge globally; however, this potential remains largely unrealized, particularly in resource-limited settings. Content is increasingly concentrated on a small number of platforms, each hosting large volumes of material compiled from diverse sources.

Objective: This scoping review aimed to identify and synthesize reported barriers and enablers to the successful design, production, and operation of open access medical education platforms, with the goal of informing strategies to improve their impact, reach, and sustainability.

Methods: We conducted a scoping review using the Arksey and O'Malley framework. A structured search was carried out on April 17, 2023, in PubMed and EBSCOhost. Citation chaining with the SnowGlobe tool and manual reference checking supplemented the search. Studies were eligible for inclusion if they examined platforms that compile content from multiple sources and reported barriers and enablers. Two reviewers (AAEA and AB) independently screened records and extracted data, with discrepancies resolved by a third reviewer (EPOF). Beginning with an a priori framework of "barriers" and "enablers," coding was then developed inductively. Thematic synthesis categorized findings by stakeholder group.

Results: Of 1108 records identified, 1064 unique records were screened, and 64 full-text papers were assessed; 34 met the inclusion criteria. The most frequently reported barriers were concerns about content-quality control, incomplete or unstructured materials, and the resources needed to sustain platforms long-term. Key enablers included the use of validated tools to assess content quality and collaboration with existing content providers and platforms to enhance visibility and learner engagement. Findings were organized into 3 stakeholder groups: learners and training programs, content designers and creators, and platform managers.

Conclusions: Open access medical education platforms have significant untapped potential to enhance global medical training. Addressing these persistent challenges-particularly around quality assurance, content organization, and sustainability-will require more structured, collaborative, and internationally coordinated approaches.

Background: Global health care education that cultivates international orientation is important for providing medical care in consideration of diverse backgrounds and collaboration with foreign medical professionals. Virtual international exchange programs could be a new type of global education in the present postpandemic era.

Objective: This study aimed to examine the effectiveness of a virtual international exchange program in fostering quality academic and professional learning and international orientation from student perspectives across 5 countries. This research is expected to contribute to education for the development of global human resources in the health professions.

Methods: This quasi-experimental study used a before-and-after design using a convergent parallel mixed methods approach. In this study, a 5-day interactive virtual program was offered to occupational therapy students from Bangladesh, Indonesia, Japan, the Philippines, and Thailand. The students were asked about their expectations and international orientation before the program, and about their evaluation of the program and international orientation afterward. Numerical data from a questionnaire on program expectations and evaluations were analyzed using descriptive statistics. Data on international orientation were subjected to qualitative analysis using steps for coding and theorization.

Results: In total, 29 students participated in the program, out of which 12 students (response ratio 41.4%) answered the research questionnaires both before and after the program. Overall, the students' expectations of the program were met in terms of expertise, scientific learning skills, and group interactions. Comparing before and after the program, mean scores of how the program met expectations increased, and the mean scores after the program in all 12 items asking about program evaluation were from 3.8 (SD 1.19) to 4.9 (SD 0.67; range: score 1 [lowest]-5 [highest]). Even though their motivation for participating in the program was not specific before the program, after the program, they reported having a more concrete image and specific form of what they learned from an international perspective. The participants enjoyed communication with others from diverse backgrounds while recognizing the difficulty of understanding different values. They also expressed satisfaction with their understanding of occupational therapy professionals and diverse societies, including medical systems from other countries.

Conclusions: Even though the analyzed sample data were small, these findings suggest that the program in this study may provide the participants with valuable opportunities. The virtual exchange program could foster students to cultivate qualities such as problem-finding or problem-solving and having interactions with groups from diverse backgrounds.