Journal of surgical education最新文献

Purpose: Residents at our training program identified timeliness of faculty feedback as an area for improvement, while faculty felt the evaluation tool was time-consuming and redundant. We sought to resolve these issues through stakeholder input and modern data analysis.

Methods: Core faculty and senior residents met to revamp the "Faculty Evaluation of Resident" end of rotation tool. The results of the most recent 5-years of evaluations were analyzed using Exploratory Factor Analysis, a dimension reduction tool provided in the IBM SPSS statistical package, to identify questions which were highly correlated. A new condensed tool was then generated by combining highly correlated questions with committee approval. Spearman's rank order correlation test was used to evaluate each new question versus the eliminated redundant questions. Time to survey completion and frequency of written feedback were compared using t-test. One-way ANOVA was then used to compare scores for each new question versus the eliminated questions that had been grouped together.

Results: 3,268 surveys were completed by 73 attendings regarding 55 resident subjects. Data were blinded by the program coordinator before analysis. Exploratory Factor Analysis indicated the initial 30-question instrument could be reduced to 12-questions, while retaining 96% of the variability in performance. The component matrix indicates that 4 areas accounted for the most variability in resident performance: Overall Performance, Communication, Operative Skill, and Systems Based Practice. Following implementation of the new evaluation form, attending surgeons completed resident evaluations at a median of 36 days after the rotation (IQR 18-59). Faculty left written feedback more frequently (53.4% vs 40.8%, p < 0.0001). For some new questions, the resident performances were statistically different. For example, new question 9 had an average rating of 3.55 out of 5.00, while the questions it replaced averaged 3.76-3.89 (p < 0.001). For other questions, no statistically significant difference was found. For example, new question 12 and the questions it replaced all averaged 3.75 out of 5.00 (p = 0.985). Survey response rates also improved from 35% to 76% at 2 months and 89.3% to 93.5% at 6 months.

Conclusions: Faculty input and advanced statistical analysis shortened a 30-question resident evaluation tool to 12-questions while retaining 96% of the variability in resident performance. The new instrument resulted in improved response rate and increased number of written comments from attendings. Application of Exploratory Factor Analysis to resident education represents novel use of this tool in surgical education.

Objective: Compare scoring outcomes between interviewers blinded to scores/grades/MSPE and those with the full applicant file to evaluate the effect of blinding on interview scores and ensure applicants can be confidently evaluated when blinding is used.

Design, setting and participants: Nineteen interviewers were purposively randomized to receive a complete application or file with all information except applicant grades/MSPE/USMLE score(s) of 90 applicants prior to 218 interviews during 2022 to 2023 general surgery recruitment. Blinding was randomly assigned while ensuring blinded and nonblinded interviews for both interviewers and applicants. Two program leaders involved in study implementation were excluded from blinding. All other aspects of the selection process remained unchanged from historic methods. Each applicant had 3 to 4 interviews. Each interview was scored prior to discussion with other faculty using a 10-point scale. Descriptive and univariate statistics analyzed scoring patterns. Qualitative data regarding the experiences of blinded interviewers was analyzed to generate themes.

Results: There were no differences in interview scores or difference from the applicants' mean scores between blinding groups. This remained true for within-applicant analyses and for all but 1 interviewer (95%) for within-interviewer analyses. Between-interviewer score differences were seen for interview scores across all interviewers and when comparing nonblinded vs. nonblinded scores across interviewers, but not when comparing blinded vs. blinded scores across interviewers. Qualitative data support the ability to confidently evaluate interview performance when blinded, frequent practice of "self-blinding" to limit bias even when given scores/grades/MSPE, and belief that scores/grades/MSPE are relevant for screening, but the interview has separate priorities.

Conclusions: Blinding of interviewers to scores/grades/MSPE did not significantly change interview scoring outcomes. Interviewer experiences support the ability to confidently evaluate interview performance when blinded. Given that negative effects of blinding were not found and prior work supports that bias may be mitigated by blinded interviews, we support its use in residency recruitment.

Objective: With the advent of virtual interviews, the potential for interview hoarding by applicants became of greater concern due to lack of financial constraints associated with in-person interviewing. Simultaneously, the average number of applications submitted each year is rising. Currently there is no cap to the number of applications or interviews an applicant may complete when applying to residency, with the exception of ophthalmology with a cap of 15 interviews. No studies have assessed the applicants' perspectives on an application or interview cap. We assessed the attitudes of surgical subspecialty applicants towards capping, which may be useful when considering innovations in residency selection.

Design/setting/participants: About 1841 applicants to the Johns Hopkins' ophthalmology, urology, plastic surgery, and orthopedic surgery residency programs from the 2022-2023 cycle were invited to respond to a 22-item questionnaire. Statistical analyses of aggregate data were conducted using R.

Results: Of the 776/1841 (42%) responses, 288 (40%) were in support of an application cap, while 455 (63%) were in support of an interview cap. Specialty (p < 0.001), gender (p < 0.001), taking a gap year (p = 0.02), medical school region (p = 0.04), and number of interviews accepted off of a waitlist (p = 0.01) were all significantly associated with a difference in opinion regarding an application cap. Specialty (p < 0.001), USMLE Step 1 score (p = 0.004), number of interviews (p < 0.001), and number of programs ranked (p < 0.001) were all significantly associated with a difference in opinion regarding an interview cap. Of those applicants who were in support of the respective caps they believed that on average a cap should consist of 48.1 (16.1) applications and 16.0 (8.0) interviews.

Conclusions: Our findings highlight the desire for interview caps among the majority of applicants to surgical subspecialties and thus this innovation may be considered by other specialties in the era of virtual interviews.