Assessing ChatGPT responses to frequently asked questions regarding total shoulder arthroplasty

Abstract

Background

“Dr. Google” has long been a resource for health information-seeking individuals. With the well-established presence of artificial intelligence in the healthcare world, it is reasonable to imagine that ChatGPT, an artificial intelligence-powered online chatbot, could become the next outlet for seeking medical advice online. Similar to Mika et al, this study aims to analyze the ChatGPT’s ability to answer frequently asked questions (FAQs) regarding total shoulder arthroplasty (TSA).

Methods

Ten FAQs regarding TSA were presented to ChatGPT and initial responses were recorded and analyzed against evidence-based literature. Responses were rated as “excellent response requiring no clarification,” “satisfactory response requiring minimal clarification,” “satisfactory response requiring moderate clarification,” or “unsatisfactory response requiring substantial clarification.”

Results

Only one response from ChatGPT was rated unsatisfactory and required substantial clarification. While no responses received an excellent rating, the average rating was considered to only require minimal or moderate clarification.

Conclusion

ChatGPT was able to provide largely accurate responses to FAQs regarding TSA while appropriately reiterating the importance of always consulting a medical professional. ChatGPT could prove to be another avenue for supplementary medical information regarding TSA.