Chat generative pre-trained transformer’s performance on dermatology-specific questions and its implications in medical education

Abstract

Background: Large language models (LLMs) like chat generative pre-trained transformer (ChatGPT) have gained popularity in healthcare by performing at or near the passing threshold for the United States Medical Licensing Exam (USMLE), but some limitations should be considered. Dermatology is a specialized medical field that relies heavily on visual recognition and images for diagnosis. This paper aimed to measure ChatGPT’s abilities to answer dermatology questions and compare this sub-specialty accuracy to its overall scores on USMLE Step exams.