Is artificial intelligence successful in the Turkish neurology board exam?

IF 1.5 4区医学 Q3 CLINICAL NEUROLOGY Neurological Research Pub Date : 2025-05-01 Epub Date: 2025-03-20 DOI:10.1080/01616412.2025.2481444

Ayse Betul Acar, Ece Yanik, Emine Altin, Ozlem Kurtkaya Kocak

{"title":"Is artificial intelligence successful in the Turkish neurology board exam?","authors":"Ayse Betul Acar, Ece Yanik, Emine Altin, Ozlem Kurtkaya Kocak","doi":"10.1080/01616412.2025.2481444","DOIUrl":null,"url":null,"abstract":"Objectives: OpenAI declared that GPT-4 performed better in academic and certain specialty areas. Medical licensing exams assess the clinical competence of doctors. We aimed to investigate for the first time howChatGPT will perform in the Turkish Neurology Proficiency Exam.Methods: TheGPT-4 version of ChatGPT was used in the study due to the presence ofimage-based questions. The multiple-choice sections of the TurkishNeurology Proficiency Exams conducted by the Turkish NeurologyAssociation (TND) in 2021, 2022 and 2023 were applied to ChatGPT-4.Questions and multiple-choice answers were used in their originalTurkish forms in the official national examination standards.Results: The success rate in all three exams ranges from 79% to 82%. There were common and different mistakes in the two trials. When the incorrect answers were re-evaluated, the correct answers were obtained.Conclusions: This is the first study to investigate the performance of ChatGPT on the real Neurology Proficiency Examination. The success rate was shown to be above GPT-3.5. Furthermore, this study showed that translating questions from the original language into English did not affect the performance of GPT-4 in medical licensing exams, unlike GPT-3.5. It is therefore very important that the information obtained is accurate and verifiable. ChatGPT-4's ability to find the correct answer with feedback on questions that it initially answered incorrectly may be due to the model's ability to generate flexible and adaptive answers.These models should be used carefully and consciously, knowing that they will not always give the correct answer.","PeriodicalId":19131,"journal":{"name":"Neurological Research","volume":" ","pages":"402-405"},"PeriodicalIF":1.5000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurological Research","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1080/01616412.2025.2481444","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/3/20 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives: OpenAI declared that GPT-4 performed better in academic and certain specialty areas. Medical licensing exams assess the clinical competence of doctors. We aimed to investigate for the first time howChatGPT will perform in the Turkish Neurology Proficiency Exam.

Methods: TheGPT-4 version of ChatGPT was used in the study due to the presence ofimage-based questions. The multiple-choice sections of the TurkishNeurology Proficiency Exams conducted by the Turkish NeurologyAssociation (TND) in 2021, 2022 and 2023 were applied to ChatGPT-4.Questions and multiple-choice answers were used in their originalTurkish forms in the official national examination standards.

Results: The success rate in all three exams ranges from 79% to 82%. There were common and different mistakes in the two trials. When the incorrect answers were re-evaluated, the correct answers were obtained.

Conclusions: This is the first study to investigate the performance of ChatGPT on the real Neurology Proficiency Examination. The success rate was shown to be above GPT-3.5. Furthermore, this study showed that translating questions from the original language into English did not affect the performance of GPT-4 in medical licensing exams, unlike GPT-3.5. It is therefore very important that the information obtained is accurate and verifiable. ChatGPT-4's ability to find the correct answer with feedback on questions that it initially answered incorrectly may be due to the model's ability to generate flexible and adaptive answers.These models should be used carefully and consciously, knowing that they will not always give the correct answer.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

人工智能在土耳其神经学委员会考试中成功了吗？

目的：OpenAI宣布GPT-4在学术和某些专业领域表现更好。行医执照考试评估医生的临床能力。我们的目的是首次调查chatgpt在土耳其神经学水平考试中的表现。方法：由于存在基于图像的问题，研究中使用了gpt -4版本的ChatGPT。土耳其神经学协会（TND）于2021年、2022年和2023年举办的土耳其神经学水平考试的多项选择部分适用于ChatGPT-4。在官方的国家考试标准中，问题和选择题的答案仍以其原始的土耳其形式使用。结果：三种检查的成功率为79% ~ 82%。在两次试验中，既有共同的错误，也有不同的错误。当不正确的答案被重新评估时，得到正确的答案。结论：本研究首次考察了ChatGPT在真正的神经学水平考试中的表现。成功率在GPT-3.5以上。此外，本研究表明，与GPT-3.5不同，将原语言的问题翻译成英语并不影响GPT-4在医学执照考试中的表现。因此，获得的信息必须准确和可核实，这一点非常重要。ChatGPT-4能够在最初回答错误的问题上通过反馈找到正确答案，这可能是由于该模型能够生成灵活和自适应的答案。这些模型应该谨慎而有意识地使用，要知道它们不会总是给出正确的答案。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Neurological Research 医学-临床神经学

CiteScore

3.60

自引率

0.00%

发文量

116

审稿时长

5.3 months

期刊介绍： Neurological Research is an international, peer-reviewed journal for reporting both basic and clinical research in the fields of neurosurgery, neurology, neuroengineering and neurosciences. It provides a medium for those who recognize the wider implications of their work and who wish to be informed of the relevant experience of others in related and more distant fields. The scope of the journal includes: •Stem cell applications •Molecular neuroscience •Neuropharmacology •Neuroradiology •Neurochemistry •Biomathematical models •Endovascular neurosurgery •Innovation in neurosurgery.