Natural Language Processing Identifies Under-Documentation of Symptoms in Patients on Hemodialysis.

Abstract

Background: Patients on hemodialysis (HD) have a high burden of emotional and physical symptoms. These symptoms are often under-recognized. NLP can be used to identify patient symptoms from the EHR. However, whether symptom documentation matches patient reported burden is unclear.

Methods: We conducted a prospective study of patients seen at an ambulatory nephrology practice from September 2020 to April 2021. We collected symptom surveys from patients, nurses, and physicians. We then developed a natural language processing (NLP) algorithm to identify symptoms from the patients' electronic health records (EHR) and validated the performance of this algorithm using manual chart review and patient surveys as a reference standard. Using patient surveys as the reference standard, we compared symptom identification by 1) physicians, 2) nurses, 3) physicians or nurses, and 4) NLP.

Results: We enrolled 97 patients into our study, 63% were female, 49% were Non-Hispanic Black, and 41% were Hispanic. The most common symptoms reported by patients were fatigue (61%), cramping (59%), dry skin (53%), muscle soreness (43%), and itching (41%). Physicians and nurses significantly under-recognized patients' symptoms (sensitivity 0.51 (95% CI 0.40-0.61) and 0.63 (95% CI 0.52-0.72) respectively). Nurses were better at identifying symptoms when patients reported more severe symptoms. There was no difference in results by patients' sex or ethnicity. NLP had a sensitivity of 0.92, specificity of 0.95, PPV of 0.75, and NPV of 0.99 with manual EHR review as the reference standard, and a sensitivity of 0.58 (95% CI 0.47-0.68), specificity of 0.73 (95% CI 0.48-0.89), PPV of 0.92 (95% CI 0.82-0.97), and NPV of 0.24 (95% CI 0.14-0.38) compared with patient surveys.

Conclusions: While patients on HD report high prevalence of symptoms, symptoms are under-recognized and under-documented. NLP was accurate at identifying symptoms when they were documented. Larger studies in representative populations are needed to assess the generalizability of the results of the study.