Leveraging ChatGPT to Produce Patient Education Materials for Common Hand Conditions

Purpose

Many adults in the United States possess basic or below basic health literacy skills, making it essential for patient education materials (PEMs) to be presented at or below a sixth-grade reading level. We evaluate the readability of PEMs generated by ChatGPT 3.5 and 4.0 for common hand conditions.

Methods

We used Chat Generative Pre-Trained Transformer (ChatGPT) 3.5 and 4.0 to generate PEMs for 50 common hand pathologies. Two consistent questions were asked to minimize variability: 1. “Please explain [Condition] to a patient at a sixth-grade reading level, including details on anatomy, symptoms, doctors' examination, and treatment (both surgical and nonsurgical).” 2. “Create a detailed patient information sheet for the general patient population at a sixth-grade reading level explaining [Condition], including points such as anatomy, symptoms, physical examination, and treatment (both surgical and nonsurgical).” Before asking the second question, a priming phase was conducted where ChatGPT 3.5 and 4.0 were presented with a text sample written at a sixth-grade reading level and informed that this was the desired output level. Multiple readability tests were used to evaluate the output, with a consensus reading level created from the results of all eight readability scores. Statistical analyses were performed using SAS 9.4.

Results

ChatGPT 4.0 successfully produced 28% of its responses at the appropriate reading level following the priming phase, compared to none by ChatGPT 3.5. ChatGPT 4.0 showed superior performance across all readability metrics.

Conclusions

ChatGPT 4.0 is a more effective tool than ChatGPT 3.5 for generating PEMs at a sixth-grade reading level for common hand conditions.

Clinical relevance

The results suggest that Artificial Intelligence could significantly enhance patient education and health literacy with further refinement.