Assessing the Efficacy of ChatGPT Prompting Strategies in Enhancing Thyroid Cancer Patient Education: A Prospective Study.

Abstract

With the rise of AI platforms, patients increasingly use them for information, relying on advanced language models like ChatGPT for answers and advice. However, the effectiveness of ChatGPT in educating thyroid cancer patients remains unclear. We designed 50 questions covering key areas of thyroid cancer management and generated corresponding responses under four different prompt strategies. These answers were evaluated based on four dimensions: accuracy, comprehensiveness, human care, and satisfaction. Additionally, the readability of the responses was assessed using the Flesch-Kincaid grade level, Gunning Fog Index, Simple Measure of Gobbledygook, and Fry readability score. We also statistically analyzed the references in the responses generated by ChatGPT. The type of prompt significantly influences the quality of ChatGPT's responses. Notably, the "statistics and references" prompt yields the highest quality outcomes. Prompts tailored to a "6th-grade level" generated the most easily understandable text, whereas responses without specific prompts were the most complex. Additionally, the "statistics and references" prompt produced the longest responses while the "6th-grade level" prompt resulted in the shortest. Notably, 87.84% of citations referenced published medical literature, but 12.82% contained misinformation or errors. ChatGPT demonstrates considerable potential for enhancing the readability and quality of thyroid cancer patient education materials. By adjusting prompt strategies, ChatGPT can generate responses that cater to diverse patient needs, improving their understanding and management of the disease. However, AI-generated content must be carefully supervised to ensure that the information it provides is accurate.