Enabling Communication for Locked-in Syndrome Patients using Deep Learning and an Emoji-based Brain Computer Interface

Abstract

Locked-in syndrome describes a condition in which patients are incapable of speaking or moving, although they do retain their cognitive capabilities. In this paper, we propose a novel Brain Computer Interface design using a versatile emoji-based symbol display and a deep learning solution to enable these patients to communicate using recordings obtained through electroencephalography (EEG). EEG signals are converted into images representing their spatiotemporal characteristics. Images are then classified using a deep convolutional neural network (CNN) to recognize the intended emoji symbol. A prototype of the proposed system was tested on five healthy volunteers, showing significant improvement in the recognition rate when compared to the classic LDA classifier.