Steven R Cooperman, Abisola Olaniyan, Roberto A Brandão
{"title":"足踝外科研究中的人工智能识别：调查研究。","authors":"Steven R Cooperman, Abisola Olaniyan, Roberto A Brandão","doi":"10.1016/j.fas.2024.10.001","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>This study evaluated the ability to differentiate between AI-generated and human-authored abstracts in foot and ankle surgery.</p><p><strong>Methods: </strong>An AI system (ChatGPT 3.0) was trained on 21 published abstracts to create six novel case abstracts. Nine foot and ankle surgeons participated in a blinded survey, tasked with distinguishing AI-generated from human-written abstracts, rating their confidence in their responses. Surveys were completed twice at two different time points to evaluate intra-/inter-observer reliability.</p><p><strong>Results: </strong>The overall accuracy rate for distinguishing AI-generated from human-written abstracts was 50.5 % (n = 109/216), indicating no better performance than random chance. Reviewer experience and AI familiarity did not significantly affect accuracy. Inter-rater reliability was moderate initially but decreased over time, and intra-rater reliability was poor.</p><p><strong>Conclusions: </strong>In their current form, AI-generated abstracts are nearly indistinguishable from human-written ones, posing challenges for consistent identification in foot and ankle surgery.</p><p><strong>Level of evidence: </strong>IV.</p>","PeriodicalId":48743,"journal":{"name":"Foot and Ankle Surgery","volume":null,"pages":null},"PeriodicalIF":1.9000,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"AI discernment in foot and ankle surgery research: A survey investigation.\",\"authors\":\"Steven R Cooperman, Abisola Olaniyan, Roberto A Brandão\",\"doi\":\"10.1016/j.fas.2024.10.001\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>This study evaluated the ability to differentiate between AI-generated and human-authored abstracts in foot and ankle surgery.</p><p><strong>Methods: </strong>An AI system (ChatGPT 3.0) was trained on 21 published abstracts to create six novel case abstracts. Nine foot and ankle surgeons participated in a blinded survey, tasked with distinguishing AI-generated from human-written abstracts, rating their confidence in their responses. Surveys were completed twice at two different time points to evaluate intra-/inter-observer reliability.</p><p><strong>Results: </strong>The overall accuracy rate for distinguishing AI-generated from human-written abstracts was 50.5 % (n = 109/216), indicating no better performance than random chance. Reviewer experience and AI familiarity did not significantly affect accuracy. Inter-rater reliability was moderate initially but decreased over time, and intra-rater reliability was poor.</p><p><strong>Conclusions: </strong>In their current form, AI-generated abstracts are nearly indistinguishable from human-written ones, posing challenges for consistent identification in foot and ankle surgery.</p><p><strong>Level of evidence: </strong>IV.</p>\",\"PeriodicalId\":48743,\"journal\":{\"name\":\"Foot and Ankle Surgery\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.9000,\"publicationDate\":\"2024-10-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Foot and Ankle Surgery\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1016/j.fas.2024.10.001\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ORTHOPEDICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Foot and Ankle Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.fas.2024.10.001","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ORTHOPEDICS","Score":null,"Total":0}
AI discernment in foot and ankle surgery research: A survey investigation.
Background: This study evaluated the ability to differentiate between AI-generated and human-authored abstracts in foot and ankle surgery.
Methods: An AI system (ChatGPT 3.0) was trained on 21 published abstracts to create six novel case abstracts. Nine foot and ankle surgeons participated in a blinded survey, tasked with distinguishing AI-generated from human-written abstracts, rating their confidence in their responses. Surveys were completed twice at two different time points to evaluate intra-/inter-observer reliability.
Results: The overall accuracy rate for distinguishing AI-generated from human-written abstracts was 50.5 % (n = 109/216), indicating no better performance than random chance. Reviewer experience and AI familiarity did not significantly affect accuracy. Inter-rater reliability was moderate initially but decreased over time, and intra-rater reliability was poor.
Conclusions: In their current form, AI-generated abstracts are nearly indistinguishable from human-written ones, posing challenges for consistent identification in foot and ankle surgery.
期刊介绍：
Foot and Ankle Surgery is essential reading for everyone interested in the foot and ankle and its disorders. The approach is broad and includes all aspects of the subject from basic science to clinical management. Problems of both children and adults are included, as is trauma and chronic disease. Foot and Ankle Surgery is the official journal of European Foot and Ankle Society.
The aims of this journal are to promote the art and science of ankle and foot surgery, to publish peer-reviewed research articles, to provide regular reviews by acknowledged experts on common problems, and to provide a forum for discussion with letters to the Editors. Reviews of books are also published. Papers are invited for possible publication in Foot and Ankle Surgery on the understanding that the material has not been published elsewhere or accepted for publication in another journal and does not infringe prior copyright.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
