Use of Deep Learning to Identify Peripheral Arterial Disease Cases From Narrative Clinical Notes

Introduction

Peripheral arterial disease (PAD) is the leading cause of amputation in the United States. Despite affecting 8.5 million Americans and more than 200 million people globally, there are significant gaps in awareness by both patients and providers. Ongoing efforts to raise PAD awareness among both the public and health-care professionals have not met widespread success. Thus, there is a need for alternative methods for identifying PAD patients. One potentially promising strategy leverages natural language processing (NLP) to digitally screen patients for PAD. Prior approaches have applied keyword search (KWS) to billing codes or unstructured clinical narratives to identify patients with PAD. However, KWS is limited by its lack of flexibility, the need for manual algorithm development, inconsistent validation, and an inherent failure to capture patients with undiagnosed PAD. Recent advances in deep learning (DL) allow modern NLP models to learn a conceptual representation of the verbiage associated with PAD. This capability may overcome the characteristic constraints of applying strict rule-based algorithms (i.e., searching for a disease-defining set of keywords or billing codes) to real-world clinical data. Herein, we investigate the use of DL to identify patients with PAD from unstructured notes in the electronic health record (EHR).

Methods

Using EHR data from a statewide health information exchange, we first created a dataset of all patients with diagnostic or procedural codes (International Classification of Diseases version 9 or 10 or Current Procedural Terminology) for PAD. This study population was then subdivided into training (70%) and testing (30%) cohorts. We based ground truth labels (PAD versus no PAD) on the presence of a primary diagnostic or procedural billing code for PAD at the encounter level. We implemented our KWS-based identification strategy using the currently published state-of-the-art algorithm for identifying PAD cases from unstructured EHR data. We developed a DL model using a BioMed-RoBERTa base that was fine-tuned on the training cohort. We compared the performance of the KWS algorithm to our DL model on a binary classification task (PAD versus no PAD).

Results

Our study included 484,363 encounters across 71,355 patients represented in 2,268,062 notes. For the task of correctly identifying PAD related notes in our testing set, the DL outperformed KWS on all model performance measures (Sens 0.70 versus 0.62; Spec 0.99 versus 0.94; PPV 0.82 versus 0.69; NPV 0.97 versus 0.96; Accuracy 0.96 versus 0.91; P value for all comparisons <0.001).

Conclusions

Our findings suggest that DL outperforms KWS for identifying PAD cases from clinical narratives. Future planned work derived from this project will develop models to stage patients based on clinical scoring systems.