Academic Psychiatry最新文献

Objective: This study was designed to investigate if artificial intelligence (AI) detection software can determine the use of AI in personal statements for residency applications.

Method: Previously written personal statements were collected from physicians who had already matched to residency through the Electronic Residency Application System. Physicians were recruited for the study through collegial relationships and were given study information via email. The study team constructed five parallel statements from the shared personal statements to prompt AI to create a personal statement of similar content. An online AI detection tool, GPTZero, was used to assess all the personal statements. Statistical analyses were conducted using R. Descriptive statistics, t-tests, and Pearson correlations were used to assess the data.

Results: Eight physicians' statements were compared to eight AI-generated statements. GPTZero was able to correctly identify AI-generated writing, assigning them significantly higher AI probability scores compared to human-authored essays. Human-generated statements were considered more readable, used shorter words with fewer syllables, and had more sentences compared to AI-generated essays. Longer average sentence length, low readability scores, and high SAT word percentages were strongly associated with AI-generated essays.

Conclusions: This study shows the capacity of GPTZero to distinguish human-created versus AI-generated writing. Use of AI can pose significant ethical challenges and carries a risk of inadvertent harm to certain applicants and erosion of trust in the application process. Authors suggest standardization of protocol regarding the use of AI prior to its integration in post-graduate medical education.

Objective: Journal clubs are a common element of many residency programs. However, there is little data showing that they improve residents' ability to interpret and use the scientific literature. The authors developed a structured journal club curriculum intended to improve scientific literacy skills among residents and increase acceptability and participation in meaningful discussion of high-quality articles among both residents and facilitators. This study examined the impact of the curriculum on objective skills, subjective skills, and engagement.

Methods: Psychiatry residents from three academic medical centers completed 10-12 virtual journal clubs during academic year 2022-2023. At the start and end of that year, residents completed a knowledge assessment and a self-rating of scientific literacy skills; at the end of the year, residents self-rated participation in, preparation for, and attitudes towards journal club sessions and assessment of scientific literacy skills.

Results: Twenty-seven residents completed self-reports at both time points. Performance on the objective skills assessment did not improve over the course of the year. Subjectively, residents reported increased understanding of common statistical techniques, ability to determine if techniques used in an article are appropriate, and the extent to which information learned from articles informs clinical practice. In the sample of 62 resident engagement surveys at follow-up, the journal club curriculum was well-received.

Conclusions: Use of a structured journal club curriculum was associated with improvements in self-rated scientific literacy skills among residents and had high acceptability. More work is needed to understand the impact of teaching methodology on journal club success.