Improving readability in AI-generated medical information on fragility fractures: the role of prompt wording on ChatGPT's responses.

Abstract

Understanding how the questions used when interacting with chatbots impact the readability of the generated text is essential for effective health communication. Using descriptive queries instead of just keywords during interaction with ChatGPT results in more readable and understandable answers about fragility fractures.

Purpose: Large language models like ChatGPT can enhance patients' understanding of medical information, making health decisions more accessible. Complex terms, such as "fragility fracture," can confuse patients, so presenting its medical content in plain language is crucial. This study explored whether conversational prompts improve readability and understanding compared to keyword-based prompts when generating patient-centered health information on fragility fractures.

Methods: The 32 most frequently searched keywords related to "fragility fracture" and "osteoporotic fracture" were identified using Google Trends. From this set, 24 keywords were selected based on relevance and entered sequentially into ChatGPT. Each keyword was tested with two prompt types: (1) plain language with keywords embedded and (2) keywords alone. The readability and comprehensibility of the AI-generated responses were assessed using the Flesch-Kincaid reading ease (FKRE) and Flesch-Kincaid grade level (FKGL), respectively. The scores of the responses were compared using the Mann-Whitney U test.

Results: The FKRE scores indicated significantly higher readability with plain language prompts (median 34.35) compared to keyword-only prompts (median 23.60). Similarly, the FKGL indicated a lower grade level for plain language prompts (median 12.05) versus keyword-only (median 14.50), with both differences achieving statistical significance.

Conclusion: Our findings suggest that using conversational prompts can enhance the readability of AI-generated medical information on fragility fractures. Clinicians and content creators should consider this approach when using AI for patient education to optimize comprehension.