Journal of Educational Evaluation for Health Professions最新文献

Giving constructive feedback is crucial for learners to bridge the gap between their current performance and the desired standards ofcompetence. Giving effective feedback is a skill that can be learned, practiced, and improved. Therefore, our aim was to explore modelsin clinical settings and assess their transferability to different clinical feedback encounters. We identified the 6 most common and accepted feedback models, including the Feedback Sandwich, the Pendleton Rules, the One-Minute Preceptor, the SET-GO model, the R2C2 (Rapport/Reaction/Content/Coach), and the ALOBA (Agenda Led Outcome-based Analysis) model. We present a handy resource describing their structure, strengths and weaknesses, requirements for educators and learners, and suitable feedback encountersfor use for each model. These feedback models represent practical frameworks for educators to adopt but also to adapt to their preferred style, combining and modifying them if necessary to suit their needs and context.

Purpose: Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method.

Methods: The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station.

Results: On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts.

Conclusion: The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.

Purpose: This study aimed to suggest a more suitable study design and the corresponding reporting guidelines in the papers published in the Journal of Educational Evaluation for Health Professionals from January 2021 to September 2022.

Methods: Among 59 papers published in the Journal of Educational Evaluation for Health Professionals from January 2021 to September 2022, research articles, review articles, and brief reports were selected. The followings were analyzed: first, the percentage of articles describing the study design in the title, abstracts, or methods; second, the portion of articles describing reporting guidelines; third, thetypes of study design and corresponding reporting guidelines; and fourth, the suggestion of a more suitable study design based on the study design algorithm for medical literature on interventions, systematic reviews & other review types, and epidemiological studies overview.

Results: Out of 45 articles, 44 described study designs (97.8%). Out of 44, 19 articles were suggested to be described with more suitable study designs, which mainly occurred in before-and-after studies, diagnostic research, and non-randomized trials. Of the 18 reporting guidelines mentioned, 8 (44.4%) were considered perfect. STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) was used for descriptive studies, before-and-after studies, and randomized controlled trials; however, its use should be reconsidered.

Conclusion: Some declarations of study design and reporting guidelines were suggested to be described with more suitable ones. Education and training on study design and reporting guidelines for researchers are needed, and reporting guideline policies for descriptive studies should also be implemented.

Purpose: Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.

Methods: Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.

Results: Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.

Conclusion: The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Purpose: Simulation training is increasingly popular in healthcare education, and often relies on specially designed manikins. However, it is also possible to work with actors, or simulated patients (SPs), which may provide a greater sense of realism. This study aimed to compare these 2 approaches, to ascertain which makes healthcare professionals feel most comfortable, which leads to the greatest improvement in confidence, and which is most beneficial to learning.

Methods: This study was embedded in a pre-existing multidisciplinary in situ simulation program. A multidisciplinary group of learners from a range of backgrounds—including nurses, doctors, and other allied health professionals—were asked to complete a questionnaire about their learning preferences. We collected 204 responses from 40 simulation sessions over 4 months, from September to December 2019. Of these 204 responses, 123 described using an SP and 81 described using a manikin.

Results: We found that 58% of respondents believed they would feel more comfortable working with an actor, while 17% would feel more comfortable using a manikin. Learners who used both modalities reported a significant increase in confidence (P<0.0001 for both). Participants felt that both modalities were beneficial to learning, but SPs provided significantly more benefits to learning than manikins (P<0.0001). The most common reason favoring SP-based simulation was the greater realism.

Conclusion: In scenarios that could reasonably be provided using either modality, we suggest that educators should give greater consideration to using SP-based simulation.

Purpose: It aimed to compare the use of the tele objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE.

Methods: This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia.

Results: In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference -0.277, P<0.001, effect size 0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020.

Conclusion: The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.

Purpose: This study examined how students’ academic performance changed after undergoing a transition to online learning during the coronavirus disease 2019 (COVID-19) pandemic, based on the test results of 16 integrated courses conducted in 3 semesters at Hanyang University College of Medicine in Korea.

Methods: For the 16 required courses that formed an integrated system-based curriculum running for 3 semesters, the major examinations’ raw scores were collected for each student. Percent-correct scores were used in the subsequent analysis. We used the t-test to compare grades between 2019 and 2020, and the Cohen D was calculated as a measure of effect size. The correlation of scores between courses was calculated using Pearson correlation coefficients.

Results: There was a significant decrease in scores in 2020 for 10 courses (62.5%). While most of the integrated system-based curriculum test scores showed strong correlations, with coefficients of 0.6 or higher in both 2019 and 2020, the correlation coefficients were generally higher in 2020. When students were divided into low, middle, and high achievement groups, low-achieving students consistently showed declining test scores in all 3 semesters.

Conclusion: Our findings suggest that the transition to online classes due to COVID-19 has led to an overall decline in academic performance. This overall decline, which may occur when the curriculum is centered on recorded lectures, needs to be addressed. Further, medical schools need to consider establishing a support system for the academic development of low-achieving students.

Purpose: Rehabilitation science programs utilize cognitive and non-cognitive factors to select students who can complete the didactic and clinical portions of the program and pass the licensure exam. Cognitive factors such a prior grade point average and standardized test scores are known to be predictive of academic performance, but the relationship of non-cognitive factors and performance is less clear. The purpose of this systematic review was to explore the relationship of non-cognitive factors to academic and clinical performance in rehabilitation science programs.

Methods: A search of 7 databases was conducted using the following eligibility criteria: graduate programs in physical therapy (PT), occupational therapy, speech-language pathology, United States-based programs, measurement of at least 1 non-cognitive factor, measurement of academic and/or clinical performance, and quantitative reporting of results. Articles were screened by title, abstract, and full text, and data were extracted.

Results: After the comprehensive screening, 21 articles were included in the review. Seventy-six percent of studies occurred in PT students. Grit, self-efficacy, emotional intelligence, and stress were the most commonly studied factors. Only self-efficacy, emotional intelligence, and personality traits were examined in clinical and academic contexts. The results were mixed for all non-cognitive factors. Higher grit and self-efficacy tended to be associated with better performance, while stress was generally associated with worse outcomes.

Conclusion: No single non-cognitive factor was consistently related to clinical or academic performance in rehabilitation science students. There is insufficient evidence currently to recommend the evaluation of a specific non-cognitive factor for admissions decisions.

We aimed to determine whether it was feasible to assess medical students as they completed a virtual sub-internship. Six students (out of 31 who completed an in-person sub-internship) participated in a 2-week virtual sub-internship, caring for patients remotely. Residents and attendings assessed those 6 students in 15 domains using the same assessment measures from the in-person sub-internship. Raters marked “unable to assess” in 75/390 responses (19%) for the virtual sub-internship versus 88/3,405 (2.6%) for the in-person sub-internship (P=0.01), most frequently for the virtual sub-internship in the domains of the physical examination (21, 81%), rapport with patients (18, 69%), and compassion (11, 42%). Students received complete assessments in most areas. Scores were higher for the in-person than the virtual sub-internship (4.67 vs. 4.45, P<0.01) for students who completed both. Students uniformly rated the virtual clerkship positively. Students can be assessed in many domains in the context of a virtual sub-internship.