Lena Rössler, Manfred Herrmann, Annette Wiegand, Philipp Kanzow
Background: Multiple-choice examinations are frequently employed among German dental schools. However, details regarding the used item types and applied scoring methods are lacking.
Objective: We aimed to gain an insight into the current usage of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programmes.
Methods: A paper-based 10-item questionnaire regarding the employed assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the Deans of Studies and to the Heads of Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using Fisher's exact test (P<.05).
Results: The response rate amounted to 90.0% (27/30 dental schools). All respondent dental schools employed multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70.4% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (88.9%) which accounted for the largest number of items in about half of the dental schools. Further item types (eg, conventional multiple-select items, Multiple-True-False, Pick-N) were only used by fewer dental schools (≤66.7%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit, requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (73.7%, 14/19 vs. 50.0%, 4/8). However, this difference was statistically not significant (P=.375). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (55.6%, 15/27). Not a single school employed alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (55.6%, 15/27).
Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.
{"title":"Usage of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programmes.","authors":"Lena Rössler, Manfred Herrmann, Annette Wiegand, Philipp Kanzow","doi":"10.2196/58126","DOIUrl":"10.2196/58126","url":null,"abstract":"<p><strong>Background: </strong>Multiple-choice examinations are frequently employed among German dental schools. However, details regarding the used item types and applied scoring methods are lacking.</p><p><strong>Objective: </strong>We aimed to gain an insight into the current usage of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programmes.</p><p><strong>Methods: </strong>A paper-based 10-item questionnaire regarding the employed assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the Deans of Studies and to the Heads of Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using Fisher's exact test (<i>P</i><.05).</p><p><strong>Results: </strong>The response rate amounted to 90.0% (27/30 dental schools). All respondent dental schools employed multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70.4% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (88.9%) which accounted for the largest number of items in about half of the dental schools. Further item types (eg, conventional multiple-select items, Multiple-True-False, Pick-N) were only used by fewer dental schools (≤66.7%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit, requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (73.7%, 14/19 vs. 50.0%, 4/8). However, this difference was statistically not significant (<i>P</i>=.375). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (55.6%, 15/27). Not a single school employed alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (55.6%, 15/27).</p><p><strong>Conclusions: </strong>Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.</p><p><strong>Clinicaltrial: </strong></p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":" ","pages":""},"PeriodicalIF":3.2,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140856088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Digital Skills to Improve Levels of Care and Renew Health Care Professions.","authors":"Massimo De Martinis, Lia Ginaldi","doi":"10.2196/58743","DOIUrl":"10.2196/58743","url":null,"abstract":"","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e58743"},"PeriodicalIF":3.6,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11085040/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140877479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Julien Grosjean, Frank Dufour, Arriel Benis, Jean-Marie Januel, Pascal Staccini, Stéfan Jacques Darmoni
<p><p>SaNuRN is a five-year project by the University of Rouen Normandy (URN) and the Côte d’Azur University (CAU) consortium to optimize digital health education for medical and paramedical students, professionals, and administrators. The project includes a skills framework, training modules, and teaching resources. In 2027, SaNuRN is expected to train a significant portion of the 400,000 health and paramedical professions students at the French national level. Our purpose is to give a synopsis of the SaNuRN initiative, emphasizing its novel educational methods and how they will enhance the delivery of digital health education. Our goals include showcasing SaNuRN as a comprehensive program consisting of a proficiency framework, instructional modules, and educational materials and explaining how SaNuRN is implemented in the participating academic institutions. SaNuRN is a project aimed at educating and training health-related and paramedics students in digital health. The project results from a cooperative effort between URN and CAU, covering four French departments. The project is based on the French National Referential on Digital Health (FNRDH), which defines the skills and competencies to be acquired and validated by every student in the health, paramedical, and social professions curricula. The SaNuRN team is currently adapting the existing URN and CAU syllabi to FNRDH and developing short-duration video capsules of 20 to 30 minutes to teach all the relevant material. The project aims to ensure that the largest student population earns the necessary skills, and it has developed a two-tier system involving facilitators who will enable the efficient expansion of the project’s educational outreach and support the students in learning the needed material efficiently. With a focus on real-world scenarios and innovative teaching activities integrating telemedicine devices and virtual professionals, SaNuRN is committed to enabling continuous learning for healthcare professionals in clinical practice. The SaNuRN team introduced new ways of evaluating healthcare professionals by shifting from a knowledge-based to a competencies-based evaluation, aligning with the Miller teaching pyramid and using the Objective Structured Clinical Examination and Script Concordance Test in digital health education. Drawing on the expertise of URN, CAU, and their public health and digital research laboratories and partners, the SaNuRN project represents a platform for continuous innovation, including telemedicine training and living labs with virtual and interactive professional activities. The SaNuRN project provides a comprehensive, personalized 30-hour training package for health and paramedical students, addressing all 70 FNRDH competencies. The program is enhanced using AI and NLP to create virtual patients and professionals for digital healthcare simulation. SaNuRN teaching materials are open-access. The project collaborates with academic institutions worldwide t
{"title":"Digital Health Education for the Future: The SaNuRN (Santé Numérique Rouen-Nice) Consortium's Journey.","authors":"Julien Grosjean, Frank Dufour, Arriel Benis, Jean-Marie Januel, Pascal Staccini, Stéfan Jacques Darmoni","doi":"10.2196/53997","DOIUrl":"10.2196/53997","url":null,"abstract":"<p><p>SaNuRN is a five-year project by the University of Rouen Normandy (URN) and the Côte d’Azur University (CAU) consortium to optimize digital health education for medical and paramedical students, professionals, and administrators. The project includes a skills framework, training modules, and teaching resources. In 2027, SaNuRN is expected to train a significant portion of the 400,000 health and paramedical professions students at the French national level. Our purpose is to give a synopsis of the SaNuRN initiative, emphasizing its novel educational methods and how they will enhance the delivery of digital health education. Our goals include showcasing SaNuRN as a comprehensive program consisting of a proficiency framework, instructional modules, and educational materials and explaining how SaNuRN is implemented in the participating academic institutions. SaNuRN is a project aimed at educating and training health-related and paramedics students in digital health. The project results from a cooperative effort between URN and CAU, covering four French departments. The project is based on the French National Referential on Digital Health (FNRDH), which defines the skills and competencies to be acquired and validated by every student in the health, paramedical, and social professions curricula. The SaNuRN team is currently adapting the existing URN and CAU syllabi to FNRDH and developing short-duration video capsules of 20 to 30 minutes to teach all the relevant material. The project aims to ensure that the largest student population earns the necessary skills, and it has developed a two-tier system involving facilitators who will enable the efficient expansion of the project’s educational outreach and support the students in learning the needed material efficiently. With a focus on real-world scenarios and innovative teaching activities integrating telemedicine devices and virtual professionals, SaNuRN is committed to enabling continuous learning for healthcare professionals in clinical practice. The SaNuRN team introduced new ways of evaluating healthcare professionals by shifting from a knowledge-based to a competencies-based evaluation, aligning with the Miller teaching pyramid and using the Objective Structured Clinical Examination and Script Concordance Test in digital health education. Drawing on the expertise of URN, CAU, and their public health and digital research laboratories and partners, the SaNuRN project represents a platform for continuous innovation, including telemedicine training and living labs with virtual and interactive professional activities. The SaNuRN project provides a comprehensive, personalized 30-hour training package for health and paramedical students, addressing all 70 FNRDH competencies. The program is enhanced using AI and NLP to create virtual patients and professionals for digital healthcare simulation. SaNuRN teaching materials are open-access. The project collaborates with academic institutions worldwide t","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e53997"},"PeriodicalIF":3.6,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11082434/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140872504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Marcos Rojas, Marcelo Rojas, Valentina Burgess, Javier Toro-Pérez, Shima Salehi
Background: The deployment of OpenAI's ChatGPT-3.5 and its subsequent versions, ChatGPT-4 and ChatGPT-4 With Vision (4V; also known as "GPT-4 Turbo With Vision"), has notably influenced the medical field. Having demonstrated remarkable performance in medical examinations globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile's medical licensing examinations-a critical step for medical practitioners in Chile-is less explored. This gap highlights the need to evaluate ChatGPT's adaptability to diverse linguistic and cultural contexts.
Objective: This study aims to evaluate the performance of ChatGPT versions 3.5, 4, and 4V in the EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile.
Methods: Three official practice drills (540 questions) from the University of Chile, mirroring the EUNACOM's structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The 3 ChatGPT versions were provided 3 attempts for each drill. Responses to questions during each attempt were systematically categorized and analyzed to assess their accuracy rate.
Results: All versions of ChatGPT passed the EUNACOM drills. Specifically, versions 4 and 4V outperformed version 3.5, achieving average accuracy rates of 79.32% and 78.83%, respectively, compared to 57.53% for version 3.5 (P<.001). Version 4V, however, did not outperform version 4 (P=.73), despite the additional visual capabilities. We also evaluated ChatGPT's performance in different medical areas of the EUNACOM and found that versions 4 and 4V consistently outperformed version 3.5. Across the different medical areas, version 3.5 displayed the highest accuracy in psychiatry (69.84%), while versions 4 and 4V achieved the highest accuracy in surgery (90.00% and 86.11%, respectively). Versions 3.5 and 4 had the lowest performance in internal medicine (52.74% and 75.62%, respectively), while version 4V had the lowest performance in public health (74.07%).
Conclusions: This study reveals ChatGPT's ability to pass the EUNACOM, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) have not significantly led to enhancements in performance on image-based questions. The variations in proficiency across medical fields suggest the need for more nuanced AI training. Additionally, the study underscores the importance of exploring innovative approaches to using AI to augment human cognition and enhance the learning process. Such advancements have the potential to significantly influence medical education, fostering not only knowledge acquisition but also the development of critical thinking and problem-solving skills among health care professionals.
{"title":"Exploring the Performance of ChatGPT Versions 3.5, 4, and 4 With Vision in the Chilean Medical Licensing Examination: Observational Study.","authors":"Marcos Rojas, Marcelo Rojas, Valentina Burgess, Javier Toro-Pérez, Shima Salehi","doi":"10.2196/55048","DOIUrl":"10.2196/55048","url":null,"abstract":"<p><strong>Background: </strong>The deployment of OpenAI's ChatGPT-3.5 and its subsequent versions, ChatGPT-4 and ChatGPT-4 With Vision (4V; also known as \"GPT-4 Turbo With Vision\"), has notably influenced the medical field. Having demonstrated remarkable performance in medical examinations globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile's medical licensing examinations-a critical step for medical practitioners in Chile-is less explored. This gap highlights the need to evaluate ChatGPT's adaptability to diverse linguistic and cultural contexts.</p><p><strong>Objective: </strong>This study aims to evaluate the performance of ChatGPT versions 3.5, 4, and 4V in the EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile.</p><p><strong>Methods: </strong>Three official practice drills (540 questions) from the University of Chile, mirroring the EUNACOM's structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The 3 ChatGPT versions were provided 3 attempts for each drill. Responses to questions during each attempt were systematically categorized and analyzed to assess their accuracy rate.</p><p><strong>Results: </strong>All versions of ChatGPT passed the EUNACOM drills. Specifically, versions 4 and 4V outperformed version 3.5, achieving average accuracy rates of 79.32% and 78.83%, respectively, compared to 57.53% for version 3.5 (P<.001). Version 4V, however, did not outperform version 4 (P=.73), despite the additional visual capabilities. We also evaluated ChatGPT's performance in different medical areas of the EUNACOM and found that versions 4 and 4V consistently outperformed version 3.5. Across the different medical areas, version 3.5 displayed the highest accuracy in psychiatry (69.84%), while versions 4 and 4V achieved the highest accuracy in surgery (90.00% and 86.11%, respectively). Versions 3.5 and 4 had the lowest performance in internal medicine (52.74% and 75.62%, respectively), while version 4V had the lowest performance in public health (74.07%).</p><p><strong>Conclusions: </strong>This study reveals ChatGPT's ability to pass the EUNACOM, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) have not significantly led to enhancements in performance on image-based questions. The variations in proficiency across medical fields suggest the need for more nuanced AI training. Additionally, the study underscores the importance of exploring innovative approaches to using AI to augment human cognition and enhance the learning process. Such advancements have the potential to significantly influence medical education, fostering not only knowledge acquisition but also the development of critical thinking and problem-solving skills among health care professionals.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e55048"},"PeriodicalIF":3.6,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11082432/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140856087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ana Luiza Dallora, Ewa Kazimiera Andersson, Bruna Gregory Palm, Doris Bohman, Gunilla Björling, Ludmiła Marcinowicz, Louise Stjernberg, Peter Anderberg
<p><strong>Background: </strong>The growing presence of digital technologies in health care requires the health workforce to have proficiency in subjects such as informatics. This has implications in the education of nursing students, as their preparedness to use these technologies in clinical situations is something that course administrators need to consider. Thus, students' attitudes toward technology could be investigated to assess their needs regarding this proficiency.</p><p><strong>Objective: </strong>This study aims to investigate attitudes (enthusiasm and anxiety) toward technology among nursing students and to identify factors associated with those attitudes.</p><p><strong>Methods: </strong>Nursing students at 2 universities in Sweden and 1 university in Poland were invited to answer a questionnaire. Data about attitudes (anxiety and enthusiasm) toward technology, eHealth literacy, electronic device skills, and frequency of using electronic devices and sociodemographic data were collected. Descriptive statistics were used to characterize the data. The Spearman rank correlation coefficient and Mann-Whitney U test were used for statistical inferences.</p><p><strong>Results: </strong>In total, 646 students answered the questionnaire-342 (52.9%) from the Swedish sites and 304 (47.1%) from the Polish site. It was observed that the students' technology enthusiasm (techEnthusiasm) was on the higher end of the Technophilia instrument (score range 1-5): 3.83 (SD 0.90), 3.62 (SD 0.94), and 4.04 (SD 0.78) for the whole sample, Swedish students, and Polish students, respectively. Technology anxiety (techAnxiety) was on the midrange of the Technophilia instrument: 2.48 (SD 0.96), 2.37 (SD 1), and 2.60 (SD 0.89) for the whole sample, Swedish students, and Polish students, respectively. Regarding techEnthusiasm among the nursing students, a negative correlation with age was found for the Swedish sample (P<.001; ρ<sub>Swedish</sub>=-0.201) who were generally older than the Polish sample, and positive correlations with the eHealth Literacy Scale score (P<.001; ρ<sub>all</sub>=0.265; ρ<sub>Swedish</sub>=0.190; ρ<sub>Polish</sub>=0.352) and with the perceived skill in using computer devices (P<.001; ρ<sub>all</sub>=0.360; ρ<sub>Swedish</sub>=0.341; ρ<sub>Polish</sub>=0.309) were found for the Swedish, Polish, and total samples. Regarding techAnxiety among the nursing students, a positive correlation with age was found in the Swedish sample (P<.001; ρ<sub>Swedish</sub>=0.184), and negative correlations with eHealth Literacy Scale score (P<.001; ρ<sub>all</sub>=-0.196; ρ<sub>Swedish</sub>=-0.262; ρ<sub>Polish</sub>=-0.133) and with the perceived skill in using computer devices (P<.001; ρ<sub>all</sub>=-0.209; ρ<sub>Swedish</sub>=-0.347; ρ<sub>Polish</sub>=-0.134) were found for the Swedish, Polish, and total samples and with the semester only for the Swedish sample (P<.001; ρ<sub>Swedish</sub>=-0.124). Gender differences were found regarding techAnxiety in
{"title":"Nursing Students' Attitudes Toward Technology: Multicenter Cross-Sectional Study.","authors":"Ana Luiza Dallora, Ewa Kazimiera Andersson, Bruna Gregory Palm, Doris Bohman, Gunilla Björling, Ludmiła Marcinowicz, Louise Stjernberg, Peter Anderberg","doi":"10.2196/50297","DOIUrl":"10.2196/50297","url":null,"abstract":"<p><strong>Background: </strong>The growing presence of digital technologies in health care requires the health workforce to have proficiency in subjects such as informatics. This has implications in the education of nursing students, as their preparedness to use these technologies in clinical situations is something that course administrators need to consider. Thus, students' attitudes toward technology could be investigated to assess their needs regarding this proficiency.</p><p><strong>Objective: </strong>This study aims to investigate attitudes (enthusiasm and anxiety) toward technology among nursing students and to identify factors associated with those attitudes.</p><p><strong>Methods: </strong>Nursing students at 2 universities in Sweden and 1 university in Poland were invited to answer a questionnaire. Data about attitudes (anxiety and enthusiasm) toward technology, eHealth literacy, electronic device skills, and frequency of using electronic devices and sociodemographic data were collected. Descriptive statistics were used to characterize the data. The Spearman rank correlation coefficient and Mann-Whitney U test were used for statistical inferences.</p><p><strong>Results: </strong>In total, 646 students answered the questionnaire-342 (52.9%) from the Swedish sites and 304 (47.1%) from the Polish site. It was observed that the students' technology enthusiasm (techEnthusiasm) was on the higher end of the Technophilia instrument (score range 1-5): 3.83 (SD 0.90), 3.62 (SD 0.94), and 4.04 (SD 0.78) for the whole sample, Swedish students, and Polish students, respectively. Technology anxiety (techAnxiety) was on the midrange of the Technophilia instrument: 2.48 (SD 0.96), 2.37 (SD 1), and 2.60 (SD 0.89) for the whole sample, Swedish students, and Polish students, respectively. Regarding techEnthusiasm among the nursing students, a negative correlation with age was found for the Swedish sample (P<.001; ρ<sub>Swedish</sub>=-0.201) who were generally older than the Polish sample, and positive correlations with the eHealth Literacy Scale score (P<.001; ρ<sub>all</sub>=0.265; ρ<sub>Swedish</sub>=0.190; ρ<sub>Polish</sub>=0.352) and with the perceived skill in using computer devices (P<.001; ρ<sub>all</sub>=0.360; ρ<sub>Swedish</sub>=0.341; ρ<sub>Polish</sub>=0.309) were found for the Swedish, Polish, and total samples. Regarding techAnxiety among the nursing students, a positive correlation with age was found in the Swedish sample (P<.001; ρ<sub>Swedish</sub>=0.184), and negative correlations with eHealth Literacy Scale score (P<.001; ρ<sub>all</sub>=-0.196; ρ<sub>Swedish</sub>=-0.262; ρ<sub>Polish</sub>=-0.133) and with the perceived skill in using computer devices (P<.001; ρ<sub>all</sub>=-0.209; ρ<sub>Swedish</sub>=-0.347; ρ<sub>Polish</sub>=-0.134) were found for the Swedish, Polish, and total samples and with the semester only for the Swedish sample (P<.001; ρ<sub>Swedish</sub>=-0.124). Gender differences were found regarding techAnxiety in ","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e50297"},"PeriodicalIF":3.6,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11091804/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140869670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p><strong>Background: </strong>Generally, cardiopulmonary resuscitation (CPR) skills decline substantially over time. By combining web-based self-regulated learning with hands-on practice, blended training can be a time- and resource-efficient approach enabling individuals to acquire or refresh CPR skills at their convenience. However, few studies have evaluated the effectiveness of blended CPR refresher training compared with that of the traditional method.</p><p><strong>Objective: </strong>This study investigated and compared the effectiveness of traditional and blended CPR training through 6-month and 12-month refresher sessions with CPR ability indicators.</p><p><strong>Methods: </strong>This study recruited participants aged ≥18 years from the Automated External Defibrillator Donation Project. The participants were divided into 4 groups based on the format of the CPR training and refresher training received: (1) initial traditional training (a 30-minute instructor-led, hands-on session) and 6-month traditional refresher training (Traditional6 group), (2) initial traditional training and 6-month blended refresher training (an 18-minute e-learning module; Mixed6 group), (3) initial traditional training and 12-month blended refresher training (Mixed12 group), and (4) initial blended training and 6-month blended refresher training (Blended6 group). CPR knowledge and performance were evaluated immediately after initial training. For each group, following initial training but before refresher training, a learning effectiveness assessment was conducted at 12 and 24 months. CPR knowledge was assessed using a written test with 15 multiple-choice questions, and CPR performance was assessed through an examiner-rated skill test and objectively through manikin feedback. A generalized estimating equation model was used to analyze changes in CPR ability indicators.</p><p><strong>Results: </strong>This study recruited 1163 participants (mean age 41.82, SD 11.6 years; n=725, 62.3% female), with 332 (28.5%), 270 (23.2%), 258 (22.2%), and 303 (26.1%) participants in the Mixed6, Traditional6, Mixed12, and Blended6 groups, respectively. No significant between-group difference was observed in knowledge acquisition after initial training (P=.23). All groups met the criteria for high-quality CPR skills (ie, average compression depth: 5-6 cm; average compression rate: 100-120 beats/min; chest recoil rate: >80%); however, a higher proportion (98/303, 32.3%) of participants receiving blended training initially demonstrated high-quality CPR skills. At 12 and 24 months, CPR skills had declined in all the groups, but the decline was significantly higher in the Mixed12 group, whereas the differences were not significant between the other groups. This finding indicates that frequent retraining can maintain high-quality CPR skills and that blended refresher training is as effective as traditional refresher training.</p><p><strong>Conclusions: </strong>Our findings indicate
{"title":"Effectiveness of Blended Versus Traditional Refresher Training for Cardiopulmonary Resuscitation: Prospective Observational Study.","authors":"Cheng-Yu Chien, Shang-Li Tsai, Chien-Hsiung Huang, Ming-Fang Wang, Chi-Chun Lin, Chen-Bin Chen, Li-Heng Tsai, Hsiao-Jung Tseng, Yan-Bo Huang, Chip-Jin Ng","doi":"10.2196/52230","DOIUrl":"10.2196/52230","url":null,"abstract":"<p><strong>Background: </strong>Generally, cardiopulmonary resuscitation (CPR) skills decline substantially over time. By combining web-based self-regulated learning with hands-on practice, blended training can be a time- and resource-efficient approach enabling individuals to acquire or refresh CPR skills at their convenience. However, few studies have evaluated the effectiveness of blended CPR refresher training compared with that of the traditional method.</p><p><strong>Objective: </strong>This study investigated and compared the effectiveness of traditional and blended CPR training through 6-month and 12-month refresher sessions with CPR ability indicators.</p><p><strong>Methods: </strong>This study recruited participants aged ≥18 years from the Automated External Defibrillator Donation Project. The participants were divided into 4 groups based on the format of the CPR training and refresher training received: (1) initial traditional training (a 30-minute instructor-led, hands-on session) and 6-month traditional refresher training (Traditional6 group), (2) initial traditional training and 6-month blended refresher training (an 18-minute e-learning module; Mixed6 group), (3) initial traditional training and 12-month blended refresher training (Mixed12 group), and (4) initial blended training and 6-month blended refresher training (Blended6 group). CPR knowledge and performance were evaluated immediately after initial training. For each group, following initial training but before refresher training, a learning effectiveness assessment was conducted at 12 and 24 months. CPR knowledge was assessed using a written test with 15 multiple-choice questions, and CPR performance was assessed through an examiner-rated skill test and objectively through manikin feedback. A generalized estimating equation model was used to analyze changes in CPR ability indicators.</p><p><strong>Results: </strong>This study recruited 1163 participants (mean age 41.82, SD 11.6 years; n=725, 62.3% female), with 332 (28.5%), 270 (23.2%), 258 (22.2%), and 303 (26.1%) participants in the Mixed6, Traditional6, Mixed12, and Blended6 groups, respectively. No significant between-group difference was observed in knowledge acquisition after initial training (P=.23). All groups met the criteria for high-quality CPR skills (ie, average compression depth: 5-6 cm; average compression rate: 100-120 beats/min; chest recoil rate: >80%); however, a higher proportion (98/303, 32.3%) of participants receiving blended training initially demonstrated high-quality CPR skills. At 12 and 24 months, CPR skills had declined in all the groups, but the decline was significantly higher in the Mixed12 group, whereas the differences were not significant between the other groups. This finding indicates that frequent retraining can maintain high-quality CPR skills and that blended refresher training is as effective as traditional refresher training.</p><p><strong>Conclusions: </strong>Our findings indicate","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e52230"},"PeriodicalIF":3.6,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11091803/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140858259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Artificial intelligence (AI) chatbots, such as ChatGPT-4, have shown immense potential for application across various aspects of medicine, including medical education, clinical practice, and research.
Objective: This study aimed to evaluate the performance of ChatGPT-4 in the 2023 Taiwan Audiologist Qualification Examination, thereby preliminarily exploring the potential utility of AI chatbots in the fields of audiology and hearing care services.
Methods: ChatGPT-4 was tasked to provide answers and reasoning for the 2023 Taiwan Audiologist Qualification Examination. The examination encompassed six subjects: (1) basic auditory science, (2) behavioral audiology, (3) electrophysiological audiology, (4) principles and practice of hearing devices, (5) health and rehabilitation of the auditory and balance systems, and (6) auditory and speech communication disorders (including professional ethics). Each subject included 50 multiple-choice questions, with the exception of behavioral audiology, which had 49 questions, amounting to a total of 299 questions.
Results: The correct answer rates across the 6 subjects were as follows: 88% for basic auditory science, 63% for behavioral audiology, 58% for electrophysiological audiology, 72% for principles and practice of hearing devices, 80% for health and rehabilitation of the auditory and balance systems, and 86% for auditory and speech communication disorders (including professional ethics). The overall accuracy rate for the 299 questions was 75%, which surpasses the examination's passing criteria of an average 60% accuracy rate across all subjects. A comprehensive review of ChatGPT-4's responses indicated that incorrect answers were predominantly due to information errors.
Conclusions: ChatGPT-4 demonstrated a robust performance in the Taiwan Audiologist Qualification Examination, showcasing effective logical reasoning skills. Our results suggest that with enhanced information accuracy, ChatGPT-4's performance could be further improved. This study indicates significant potential for the application of AI chatbots in audiology and hearing care services.
{"title":"Exploring the Performance of ChatGPT-4 in the Taiwan Audiologist Qualification Examination: Preliminary Observational Study Highlighting the Potential of AI Chatbots in Hearing Care.","authors":"Shangqiguo Wang, Changgeng Mo, Yuan Chen, Xiaolu Dai, Huiyi Wang, Xiaoli Shen","doi":"10.2196/55595","DOIUrl":"10.2196/55595","url":null,"abstract":"<p><strong>Background: </strong>Artificial intelligence (AI) chatbots, such as ChatGPT-4, have shown immense potential for application across various aspects of medicine, including medical education, clinical practice, and research.</p><p><strong>Objective: </strong>This study aimed to evaluate the performance of ChatGPT-4 in the 2023 Taiwan Audiologist Qualification Examination, thereby preliminarily exploring the potential utility of AI chatbots in the fields of audiology and hearing care services.</p><p><strong>Methods: </strong>ChatGPT-4 was tasked to provide answers and reasoning for the 2023 Taiwan Audiologist Qualification Examination. The examination encompassed six subjects: (1) basic auditory science, (2) behavioral audiology, (3) electrophysiological audiology, (4) principles and practice of hearing devices, (5) health and rehabilitation of the auditory and balance systems, and (6) auditory and speech communication disorders (including professional ethics). Each subject included 50 multiple-choice questions, with the exception of behavioral audiology, which had 49 questions, amounting to a total of 299 questions.</p><p><strong>Results: </strong>The correct answer rates across the 6 subjects were as follows: 88% for basic auditory science, 63% for behavioral audiology, 58% for electrophysiological audiology, 72% for principles and practice of hearing devices, 80% for health and rehabilitation of the auditory and balance systems, and 86% for auditory and speech communication disorders (including professional ethics). The overall accuracy rate for the 299 questions was 75%, which surpasses the examination's passing criteria of an average 60% accuracy rate across all subjects. A comprehensive review of ChatGPT-4's responses indicated that incorrect answers were predominantly due to information errors.</p><p><strong>Conclusions: </strong>ChatGPT-4 demonstrated a robust performance in the Taiwan Audiologist Qualification Examination, showcasing effective logical reasoning skills. Our results suggest that with enhanced information accuracy, ChatGPT-4's performance could be further improved. This study indicates significant potential for the application of AI chatbots in audiology and hearing care services.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e55595"},"PeriodicalIF":3.6,"publicationDate":"2024-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11067446/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140872544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lena Rettinger, Peter Putz, Lea Aichinger, Susanne Maria Javorszky, Klaus Widhalm, Veronika Ertelt-Bach, Andreas Huber, Sevan Sargis, Lukas Maul, Oliver Radinger, Franz Werner, Sebastian Kuhn
[This corrects the article DOI: 10.2196/51112.].
[此处更正了文章 DOI：10.2196/51112]。
{"title":"Correction: Telehealth Education in Allied Health Care and Nursing: Web-Based Cross-Sectional Survey of Students' Perceived Knowledge, Skills, Attitudes, and Experience.","authors":"Lena Rettinger, Peter Putz, Lea Aichinger, Susanne Maria Javorszky, Klaus Widhalm, Veronika Ertelt-Bach, Andreas Huber, Sevan Sargis, Lukas Maul, Oliver Radinger, Franz Werner, Sebastian Kuhn","doi":"10.2196/59919","DOIUrl":"10.2196/59919","url":null,"abstract":"<p><p>[This corrects the article DOI: 10.2196/51112.].</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e59919"},"PeriodicalIF":3.6,"publicationDate":"2024-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11087849/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140858252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p><strong>Background: </strong>Carers often assume key roles in cancer care. However, many carers report feeling disempowered and ill-equipped to support patients. Our group published evidence-based guidelines (the Triadic Oncology [TRIO] Guidelines) to improve oncology clinician engagement with carers and the management of challenging situations involving carers.</p><p><strong>Objective: </strong>To facilitate implementation of the TRIO Guidelines in clinical practice, we aimed to develop, iteratively refine, and conduct user testing of a suite of evidence-based and interactive web-based education modules for oncology clinicians (e-Triadic Oncology [eTRIO]), patients with cancer, and carers (eTRIO for Patients and Carers [eTRIO-pc]). These were designed to improve carer involvement, communication, and shared decision-making in the cancer management setting.</p><p><strong>Methods: </strong>The eTRIO education modules were based on extensive research, including systematic reviews, qualitative interviews, and consultation analyses. Guided by the person-based approach, module content and design were reviewed by an expert advisory group comprising academic and clinical experts (n=13) and consumers (n=5); content and design were continuously and iteratively refined. User experience testing (including "think-aloud" interviews and administration of the System Usability Scale [SUS]) of the modules was completed by additional clinicians (n=5), patients (n=3), and carers (n=3).</p><p><strong>Results: </strong>The final clinician module comprises 14 sections, requires approximately 1.5 to 2 hours to complete, and covers topics such as carer-inclusive communication and practices; supporting carer needs; and managing carer dominance, anger, and conflicting patient-carer wishes. The usability of the module was rated by 5 clinicians, with a mean SUS score of 75 (SD 5.3), which is interpreted as good. Clinicians often desired information in a concise format, divided into small "snackable" sections that could be easily recommenced if they were interrupted. The carer module features 11 sections; requires approximately 1.5 hours to complete; and includes topics such as the importance of carers, carer roles during consultations, and advocating for the patient. The patient module is an adaptation of the relevant carer module sections, comprising 7 sections and requiring 1 hour to complete. The average SUS score as rated by 6 patients and carers was 78 (SD 16.2), which is interpreted as good. Interactive activities, clinical vignette videos, and reflective learning exercises are incorporated into all modules. Patient and carer consumer advisers advocated for empathetic content and tone throughout their modules, with an easy-to-read and navigable module interface.</p><p><strong>Conclusions: </strong>The eTRIO suite of modules were rigorously developed using a person-based design methodology to meet the unique information needs and learning requirements of clinicians,
{"title":"Development of Web-Based Education Modules to Improve Carer Engagement in Cancer Care: Design and User Experience Evaluation of the e-Triadic Oncology (eTRIO) Modules for Clinicians, Patients, and Carers.","authors":"Rebekah Laidsaar-Powell, Sarah Giunta, Phyllis Butow, Rachael Keast, Bogda Koczwara, Judy Kay, Michael Jefford, Sandra Turner, Christobel Saunders, Penelope Schofield, Frances Boyle, Patsy Yates, Kate White, Annie Miller, Zoe Butt, Melanie Bonnaudet, Ilona Juraskova","doi":"10.2196/50118","DOIUrl":"https://doi.org/10.2196/50118","url":null,"abstract":"<p><strong>Background: </strong>Carers often assume key roles in cancer care. However, many carers report feeling disempowered and ill-equipped to support patients. Our group published evidence-based guidelines (the Triadic Oncology [TRIO] Guidelines) to improve oncology clinician engagement with carers and the management of challenging situations involving carers.</p><p><strong>Objective: </strong>To facilitate implementation of the TRIO Guidelines in clinical practice, we aimed to develop, iteratively refine, and conduct user testing of a suite of evidence-based and interactive web-based education modules for oncology clinicians (e-Triadic Oncology [eTRIO]), patients with cancer, and carers (eTRIO for Patients and Carers [eTRIO-pc]). These were designed to improve carer involvement, communication, and shared decision-making in the cancer management setting.</p><p><strong>Methods: </strong>The eTRIO education modules were based on extensive research, including systematic reviews, qualitative interviews, and consultation analyses. Guided by the person-based approach, module content and design were reviewed by an expert advisory group comprising academic and clinical experts (n=13) and consumers (n=5); content and design were continuously and iteratively refined. User experience testing (including \"think-aloud\" interviews and administration of the System Usability Scale [SUS]) of the modules was completed by additional clinicians (n=5), patients (n=3), and carers (n=3).</p><p><strong>Results: </strong>The final clinician module comprises 14 sections, requires approximately 1.5 to 2 hours to complete, and covers topics such as carer-inclusive communication and practices; supporting carer needs; and managing carer dominance, anger, and conflicting patient-carer wishes. The usability of the module was rated by 5 clinicians, with a mean SUS score of 75 (SD 5.3), which is interpreted as good. Clinicians often desired information in a concise format, divided into small \"snackable\" sections that could be easily recommenced if they were interrupted. The carer module features 11 sections; requires approximately 1.5 hours to complete; and includes topics such as the importance of carers, carer roles during consultations, and advocating for the patient. The patient module is an adaptation of the relevant carer module sections, comprising 7 sections and requiring 1 hour to complete. The average SUS score as rated by 6 patients and carers was 78 (SD 16.2), which is interpreted as good. Interactive activities, clinical vignette videos, and reflective learning exercises are incorporated into all modules. Patient and carer consumer advisers advocated for empathetic content and tone throughout their modules, with an easy-to-read and navigable module interface.</p><p><strong>Conclusions: </strong>The eTRIO suite of modules were rigorously developed using a person-based design methodology to meet the unique information needs and learning requirements of clinicians, ","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e50118"},"PeriodicalIF":3.6,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11063882/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140865976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Medical history contributes approximately 80% to a diagnosis, although physical examinations and laboratory investigations increase a physician's confidence in the medical diagnosis. The concept of artificial intelligence (AI) was first proposed more than 70 years ago. Recently, its role in various fields of medicine has grown remarkably. However, no studies have evaluated the importance of patient history in AI-assisted medical diagnosis.
Objective: This study explored the contribution of patient history to AI-assisted medical diagnoses and assessed the accuracy of ChatGPT in reaching a clinical diagnosis based on the medical history provided.
Methods: Using clinical vignettes of 30 cases identified in The BMJ, we evaluated the accuracy of diagnoses generated by ChatGPT. We compared the diagnoses made by ChatGPT based solely on medical history with the correct diagnoses. We also compared the diagnoses made by ChatGPT after incorporating additional physical examination findings and laboratory data alongside history with the correct diagnoses.
Results: ChatGPT accurately diagnosed 76.6% (23/30) of the cases with only the medical history, consistent with previous research targeting physicians. We also found that this rate was 93.3% (28/30) when additional information was included.
Conclusions: Although adding additional information improves diagnostic accuracy, patient history remains a significant factor in AI-assisted medical diagnosis. Thus, when using AI in medical diagnosis, it is crucial to include pertinent and correct patient histories for an accurate diagnosis. Our findings emphasize the continued significance of patient history in clinical diagnoses in this age and highlight the need for its integration into AI-assisted medical diagnosis systems.
{"title":"Importance of Patient History in Artificial Intelligence-Assisted Medical Diagnosis: Comparison Study.","authors":"Fumitoshi Fukuzawa, Yasutaka Yanagita, Daiki Yokokawa, Shun Uchida, Shiho Yamashita, Yu Li, Kiyoshi Shikino, Tomoko Tsukamoto, Kazutaka Noda, Takanori Uehara, Masatomi Ikusaka","doi":"10.2196/52674","DOIUrl":"https://doi.org/10.2196/52674","url":null,"abstract":"<p><strong>Background: </strong>Medical history contributes approximately 80% to a diagnosis, although physical examinations and laboratory investigations increase a physician's confidence in the medical diagnosis. The concept of artificial intelligence (AI) was first proposed more than 70 years ago. Recently, its role in various fields of medicine has grown remarkably. However, no studies have evaluated the importance of patient history in AI-assisted medical diagnosis.</p><p><strong>Objective: </strong>This study explored the contribution of patient history to AI-assisted medical diagnoses and assessed the accuracy of ChatGPT in reaching a clinical diagnosis based on the medical history provided.</p><p><strong>Methods: </strong>Using clinical vignettes of 30 cases identified in The BMJ, we evaluated the accuracy of diagnoses generated by ChatGPT. We compared the diagnoses made by ChatGPT based solely on medical history with the correct diagnoses. We also compared the diagnoses made by ChatGPT after incorporating additional physical examination findings and laboratory data alongside history with the correct diagnoses.</p><p><strong>Results: </strong>ChatGPT accurately diagnosed 76.6% (23/30) of the cases with only the medical history, consistent with previous research targeting physicians. We also found that this rate was 93.3% (28/30) when additional information was included.</p><p><strong>Conclusions: </strong>Although adding additional information improves diagnostic accuracy, patient history remains a significant factor in AI-assisted medical diagnosis. Thus, when using AI in medical diagnosis, it is crucial to include pertinent and correct patient histories for an accurate diagnosis. Our findings emphasize the continued significance of patient history in clinical diagnoses in this age and highlight the need for its integration into AI-assisted medical diagnosis systems.</p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":"10 ","pages":"e52674"},"PeriodicalIF":3.6,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11024399/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140863279","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号