Walking forward or on hold: Could the ChatGPT be applied for seeking health information in neurosurgical settings?

Abstract

Self-management is important for patients suffering from cerebrovascular events after neurosurgical procedures. An increasing number of artificial intelligence (AI)-assisted tools have been used in postoperative health management. ChatGPT is a new trend dialog-based chatbot that could be used as a supplemental tool for seeking health information. Responses from ChatGPT version 3.5 and 4.0 toward 13 questions raised by experienced neurosurgeons were evaluated in this exploratory study for their consistency and appropriateness blindly by the other three neurosurgeons. The readability of response text was investigated quantitively by word count and the Gunning Fog and Flesch–Kincaid indices. Results showed that the chatbot could provide relatively stable output between the two versions on consistency and appropriateness (χ² = 0.348). As for readability, there was a higher demand for readers to comprehend the output text in the 4.0 version (more counts of words; lower Flesch–Kincaid reading ease score; and higher Flesch–Kincaid grade level). In general, the capacity of ChatGPT to deliver effective health information is still under debate.