Using Natural Language Processing to develop risk-tier specific suicide prediction models for Veterans Affairs patients

Abstract

Suicide is a leading cause of death. Suicide rates are particularly elevated among Department of Veterans Affairs (VA) patients. While VA has made impactful suicide prevention advances, efforts primarily target high-risk patients with documented suicide risk. This high-risk population accounts for less than 10% of VA patient suicide deaths. We previously evaluated epidemiological patterns among VA patients that had lower classified suicide risk and derived moderate- and low-risk groupings. Expanding upon VA's leading suicide prediction model, this study uses national VA data to refine high-, moderate-, and low-risk specific suicide prediction methods. We selected all VA patients who died by suicide in 2017 or 2018 (n = 4584), matching each case with five controls who remained alive during treatment year and shared suicide risk percentiles. We extracted all sample unstructured electronic health record notes, analyzed them using natural language processing, and applied machine-learning classification algorithms to develop risk-tier-specific predictive models. We calculated area under the curve (AUC) and suicide risk concentration to evaluate predictive accuracy and analyzed derived words.

Results

Our high-risk model (AUC = 0.621 (95% CI: 0.55–0.68)), moderate-risk (AUC = 0.669 (95% CI: 0.64–0.71)), and low-risk (AUC = 0.673 (95% CI: 0.63–0.72)) models offered significant predictive accuracy over VA's leading suicide prediction algorithm. Derived words varied considerably, the high-risk model including chronic condition service words, moderate-risk model including outpatient care, and low-risk model including acute condition care.

Study suggests benefit of leveraging unstructured electronic health records and expands prediction resources for non-high-risk suicide decedents, an historically underserved population.