A Review of ChatGPT as a Reliable Source of Scientific Information Regarding Endodontic Local Anesthesia

IF 3.6 2区医学 Q1 DENTISTRY, ORAL SURGERY & MEDICINE Journal of endodontics Pub Date : 2025-05-01 DOI:10.1016/j.joen.2025.02.002

Al Reader DDS, MS, Melissa Drum DDS, MS

{"title":"A Review of ChatGPT as a Reliable Source of Scientific Information Regarding Endodontic Local Anesthesia","authors":"Al Reader DDS, MS, Melissa Drum DDS, MS","doi":"10.1016/j.joen.2025.02.002","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>ChatGPT is an artificial intelligence chatbot, developed by OpenAI, which uses Deep Learning technology for information processing. The chatbot uses natural language processing and machine learning algorithms to respond to users’ questions. The purpose of this study was to review ChatGPT responses to determine if they were a reliable source of scientific information regarding local anesthesia for endodontics.</div></div><div><h3>Methods</h3><div>Sixteen representative questions pertaining to local anesthesia for endodontics were selected. ChatGPT was asked to answer the 16 questions and provide supporting references. Each provided ChatGPT reference was evaluated for accuracy using NLM NIH.GOV (PubMed), Google Scholar, journal citations, and author citations. Peer-reviewed, evidence-based literature citations related to the initial questions were collected by the authors. The 2 authors independently compared the answers of the ChatGPT to the peer-reviewed, evidence-based literature using a 5-answer Likert-type scale.</div></div><div><h3>Results</h3><div>ChatGPT was reliable 50% of the time when compared to the peer-reviewed, evidence-based literature. That is, ChatGPT had the same literature-based response as our peer-reviewed, evidence-based literature in 16 of the 32 questions. Of the 51 total references for Chatbot, 59% (30 of 51) had the wrong reference, 12% (6 of 51) of the references couldn't be retrieved and 18% (9 of 51) of the references were hallucinations (made up references).</div></div><div><h3>Conclusions</h3><div>Artificial intelligence needs further training to be trusted for accurate information in the field of endodontic anesthesia. ChatGPT should continue to improve to provide reliable information for providers and patients alike.</div></div>","PeriodicalId":15703,"journal":{"name":"Journal of endodontics","volume":"51 5","pages":"Pages 571-576"},"PeriodicalIF":3.6000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of endodontics","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0099239925000676","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"DENTISTRY, ORAL SURGERY & MEDICINE","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

ChatGPT is an artificial intelligence chatbot, developed by OpenAI, which uses Deep Learning technology for information processing. The chatbot uses natural language processing and machine learning algorithms to respond to users’ questions. The purpose of this study was to review ChatGPT responses to determine if they were a reliable source of scientific information regarding local anesthesia for endodontics.

Methods

Sixteen representative questions pertaining to local anesthesia for endodontics were selected. ChatGPT was asked to answer the 16 questions and provide supporting references. Each provided ChatGPT reference was evaluated for accuracy using NLM NIH.GOV (PubMed), Google Scholar, journal citations, and author citations. Peer-reviewed, evidence-based literature citations related to the initial questions were collected by the authors. The 2 authors independently compared the answers of the ChatGPT to the peer-reviewed, evidence-based literature using a 5-answer Likert-type scale.

Results

ChatGPT was reliable 50% of the time when compared to the peer-reviewed, evidence-based literature. That is, ChatGPT had the same literature-based response as our peer-reviewed, evidence-based literature in 16 of the 32 questions. Of the 51 total references for Chatbot, 59% (30 of 51) had the wrong reference, 12% (6 of 51) of the references couldn't be retrieved and 18% (9 of 51) of the references were hallucinations (made up references).

Conclusions

Artificial intelligence needs further training to be trusted for accurate information in the field of endodontic anesthesia. ChatGPT should continue to improve to provide reliable information for providers and patients alike.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

ChatGPT作为牙髓局部麻醉可靠的科学信息来源的综述。

简介：ChatGPT是由OpenAI开发的人工智能（AI）聊天机器人，使用深度学习（DL）技术进行信息处理。该聊天机器人使用自然语言处理（NLP）和机器学习（ML）算法来回答用户的问题。本研究的目的是回顾ChatGPT反应，以确定它们是否是牙髓学局部麻醉的可靠科学信息来源。材料与方法：选取16个与牙髓学局部麻醉相关的代表性问题。ChatGPT被要求回答这16个问题并提供支持性参考资料。使用NLM NIH.GOV（PubMed）、b谷歌Scholar、期刊引文和作者引文来评估每个提供的ChatGPT参考文献的准确性。作者收集了与最初问题相关的同行评审、循证文献引用。两位作者使用李克特式的5个答案量表，独立地将ChatGPT的答案与同行评议的循证文献进行了比较。结果：与同行评议的循证文献相比，ChatGPT的可靠性为50%。也就是说，在32个问题中，ChatGPT的回答与我们同行评议的、基于证据的文献中的16个相同。在Chatbot的51个参考文献中，59%（51个中的30个）有错误的参考文献；12%（6 / 51）的文献无法被检索；18%（51人中有9人）的参考文献是幻觉（捏造的参考文献）。结论：人工智能在牙髓麻醉领域需要进一步的培训，才能获得可靠的准确信息。ChatGPT应该继续改进，为提供者和患者提供可靠的信息。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Journal of endodontics 医学-牙科与口腔外科

CiteScore

8.80

自引率

9.50%

发文量

224

审稿时长

42 days

期刊介绍： The Journal of Endodontics, the official journal of the American Association of Endodontists, publishes scientific articles, case reports and comparison studies evaluating materials and methods of pulp conservation and endodontic treatment. Endodontists and general dentists can learn about new concepts in root canal treatment and the latest advances in techniques and instrumentation in the one journal that helps them keep pace with rapid changes in this field.