Advancing delirium classification: A clinical notes-based natural language processing-supported machine learning model

Abstract

Objective

The study of the epidemiology of delirium in hospitalized patients is challenging. We aimed to identify the presence or absence of delirium from clinical text notes using natural language processing (NLP) techniques and machine learning (ML) models.

Materials and methods

We developed a delirium predictive model using 942 clinical notes from hospitalized patients with an ICD-10 delirium hospital discharge code. Moreover, we implemented ML models using a) delirium-suggestive words from an expert-defined dictionary or b) free text in clinical notes. Both strategies considered positive and negative delirium-associated words.

Results

At the note level, for the dictionary method, the logistic regression model achieved an area under the receiver-operating curve (AUROC) of 0.917 for positive words and 0.914 for combined positive and negative words. The areas under the precision-recall curve (AUPR) were 0.893 and 0.897, respectively. For the free-text method, the model achieved an AUROC of 0.826 and 0.830 and AUPR of 0.852 and 0.856, respectively.

Discussion

NLP-based ML models accurately identified the presence of delirium in clinical notes. The dictionary-based method was superior to the free-text method. The use of negative features improved performance in both methods.

Conclusion

Our proposed NLP-based ML model identified delirium in clinical notes. This model could automatically screen millions of notes and facilitate the study of the epidemiology of in-hospital delirium.