Epilepsy surgery candidate identification with artificial intelligence: An implementation study

Abstract

Background

To (a) evaluate the effect of a machine learning algorithm in the identification of patients suitable for epilepsy surgery evaluation, and (b) examine the performance of a large language model (LLM) in the collation of key pieces of information pertaining to epilepsy surgery evaluation referral.

Methods

Artificial intelligence analyses were performed for all patients seen in the epilepsy or first seizure clinic at a tertiary hospital over a 12-month period. This study design was intended to emulate a case review that could subsequently be conducted periodically (e.g., quarterly). The previously derived random forest model was used to stratify all patients by their likelihood of being a candidate for epilepsy surgery evaluation, and the top 5% of cases underwent manual case note review. An open source LLM was utilised to answer 7 prompts summarising and extracting pieces of information from the most recent clinic note, which would be relevant to epilepsy surgery evaluation referral.

Results

310 patients were included in the study, with 15 undergoing manual review. Of these patients 8/15 (53.3 %) met the prespecified criteria for epilepsy surgery evaluation. 3/15 (20.0 %) of these patients were subsequently referred for further evaluation within 1 month of the study. The LLM had an accuracy ranging between 80 % to 100 % on the different prompts. Errors occurred most often when summarising the management plan. Errors included hallucinations, omissions, and copying erroneous information.

Conclusions

Artificial intelligence may be able to assist with the identification of patients suitable for epilepsy surgery evaluation.