Giacomo Scaioli, Giuseppina Lo Moro, Francesco Conrado, Lorenzo Rosset, Fabrizio Bert, Roberta Siliquini
{"title":"探索 ChatGPT 在临床推理和决策方面的潜力：一项关于意大利医学住院医师考试的横断面研究。","authors":"Giacomo Scaioli, Giuseppina Lo Moro, Francesco Conrado, Lorenzo Rosset, Fabrizio Bert, Roberta Siliquini","doi":"10.4415/ANN_23_04_05","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>This study aimed to assess the performance of ChatGPT, a large language model (LLM), on the Italian State Exam for Medical Residency (SSM) test to determine its potential as a tool for medical education and clinical decision-making support.</p><p><strong>Materials and methods: </strong>A total of 136 questions were obtained from the official SSM test. ChatGPT responses were analyzed and compared to the performance of medical doctors who took the test in 2022. Questions were classified into clinical cases (CC) and notional questions (NQ).</p><p><strong>Results: </strong>ChatGPT achieved an overall accuracy of 90.44%, with higher performance on clinical cases (92.45%) than on notional questions (89.15%). Compared to medical doctors' scores, ChatGPT performance was higher than 99.6% of the participants.</p><p><strong>Conclusions: </strong>These results suggest that ChatGPT holds promise as a valuable tool in clinical decision-making, particularly in the context of clinical reasoning. Further research is needed to explore the potential applications and implementation of large language models (LLMs) in medical education and medical practice.</p>","PeriodicalId":8246,"journal":{"name":"Annali dell'Istituto superiore di sanita","volume":null,"pages":null},"PeriodicalIF":1.1000,"publicationDate":"2023-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Exploring the potential of ChatGPT for clinical reasoning and decision-making: a cross-sectional study on the Italian Medical Residency Exam.\",\"authors\":\"Giacomo Scaioli, Giuseppina Lo Moro, Francesco Conrado, Lorenzo Rosset, Fabrizio Bert, Roberta Siliquini\",\"doi\":\"10.4415/ANN_23_04_05\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>This study aimed to assess the performance of ChatGPT, a large language model (LLM), on the Italian State Exam for Medical Residency (SSM) test to determine its potential as a tool for medical education and clinical decision-making support.</p><p><strong>Materials and methods: </strong>A total of 136 questions were obtained from the official SSM test. ChatGPT responses were analyzed and compared to the performance of medical doctors who took the test in 2022. Questions were classified into clinical cases (CC) and notional questions (NQ).</p><p><strong>Results: </strong>ChatGPT achieved an overall accuracy of 90.44%, with higher performance on clinical cases (92.45%) than on notional questions (89.15%). Compared to medical doctors' scores, ChatGPT performance was higher than 99.6% of the participants.</p><p><strong>Conclusions: </strong>These results suggest that ChatGPT holds promise as a valuable tool in clinical decision-making, particularly in the context of clinical reasoning. Further research is needed to explore the potential applications and implementation of large language models (LLMs) in medical education and medical practice.</p>\",\"PeriodicalId\":8246,\"journal\":{\"name\":\"Annali dell'Istituto superiore di sanita\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.1000,\"publicationDate\":\"2023-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annali dell'Istituto superiore di sanita\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.4415/ANN_23_04_05\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annali dell'Istituto superiore di sanita","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.4415/ANN_23_04_05","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH","Score":null,"Total":0}
Exploring the potential of ChatGPT for clinical reasoning and decision-making: a cross-sectional study on the Italian Medical Residency Exam.
Background: This study aimed to assess the performance of ChatGPT, a large language model (LLM), on the Italian State Exam for Medical Residency (SSM) test to determine its potential as a tool for medical education and clinical decision-making support.
Materials and methods: A total of 136 questions were obtained from the official SSM test. ChatGPT responses were analyzed and compared to the performance of medical doctors who took the test in 2022. Questions were classified into clinical cases (CC) and notional questions (NQ).
Results: ChatGPT achieved an overall accuracy of 90.44%, with higher performance on clinical cases (92.45%) than on notional questions (89.15%). Compared to medical doctors' scores, ChatGPT performance was higher than 99.6% of the participants.
Conclusions: These results suggest that ChatGPT holds promise as a valuable tool in clinical decision-making, particularly in the context of clinical reasoning. Further research is needed to explore the potential applications and implementation of large language models (LLMs) in medical education and medical practice.
期刊介绍：
Annali dell’Istituto Superiore di Sanità is a peer reviewed quarterly science journal which publishes research articles in biomedicine, translational research and in many other disciplines of the health sciences. The journal includes the following material: original articles, reviews, commentaries, editorials, brief and technical notes, book reviews.
The publication of Monographic Sections has been discontinued. In case you wish to present a small number of coordinated contributions on specific themes concerning priorities in public health, please contact the Editorial office.
The journal is in English.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
