Development of a natural language processing algorithm for the detection of spinal metastasis based on magnetic resonance imaging reports

Abstract

Background

Metastasis to the spinal column is a common complication of malignancy, potentially causing pain and neurologic injury. An automated system to identify and refer patients with spinal metastases can help overcome barriers to timely treatment. We describe the training, optimization and validation of a natural language processing algorithm to identify the presence of vertebral metastasis and metastatic epidural cord compression (MECC) from radiology reports of spinal MRIs.

Methods

Reports from patients with spine MRI studies performed between January 1, 2008 and April 14, 2019 were reviewed by a team of radiologists to assess for the presence of cancer and generate a labeled dataset for model training. Using regular expression, impression sections were extracted from the reports and converted to all lower-case letters with all nonalphabetic characters removed. The reports were then tokenized and vectorized using the doc2vec algorithm. These were then used to train a neural network to predict the likelihood of spinal tumor or MECC. For each report, the model provided a number from 0 to 1 corresponding to its impression. We then obtained 111 MRI reports from outside the test set, 92 manually labeled negative and 19 with MECC to test the model's performance.

Results

About 37,579 radiology reports were reviewed. About 36,676 were labeled negative, and 903 with MECC. We chose a cutoff of 0.02 as a positive result to optimize for a low false negative rate. At this threshold we found a 100% sensitivity rate with a low false positive rate of 2.2%.

Conclusions

The NLP model described predicts the presence of spinal tumor and MECC in spine MRI reports with high accuracy. We plan to implement the algorithm into our EMR to allow for faster referral of these patients to appropriate specialists, allowing for reduced morbidity and increased survival.