Comparative assessment of three AI platforms in answering USMLE Step 1 anatomy questions or identifying anatomical structures on radiographs.

Abstract

The application of artificial intelligence (AI) in education has gained great attention recently. Integration of AI tools in anatomy teaching is currently engaging researchers and academics worldwide. Several AI chatbots have been generated, the most popular being ChatGPT (OpenAI: San Francisco, California, USA). Since its first public release in November 2022, several research papers have pointed to its potential role in anatomy education. However, it is not yet known whether it will prove superior to other available AI tools in this role. This article sheds some light on the current status of research concerning AI applications in anatomy education and compares the performances of three well-known chatbots (ChatGPT, Gemini, and Claude) in answering anatomy questions. A total of 23 questions were used as prompts for each chatbot. These questions comprised 10 knowledge-based, 10 analysis-based USMLE Step 1-type, and three radiographs. ChatGPT was the most accurate of the three, scoring 100% accuracy. However, in terms of comprehensiveness, Claude was the best; it gave very organized anatomical responses. Gemini performed less well than the other two, with a scored accuracy of 60% and less scientific explanations. On the basis of these findings, this study recommends the incorporation of Claude and ChatGPT in anatomy education, but not Gemini, at least in its current state.