A smart virtual keyboard to improve communication of locked-in patients

Abstract

Over the last decade, thanks to advances in sensor technology, imaging, and data analytics, the use of AI-based medical devices has seen an exponential increase, with massive adoption by hospitals and specialized laboratories. Among support devices for patients with limited or no residual mobility, and unable to interact to express their basic needs, artificial intelligence technologies offer promising solutions for augmentative and alternative communication (AAC), necessary for people with speech disorders, language impairment, and autism. This paper focuses on the development of a smart virtual keyboard for patients with reduced communication capabilities — specifically improving the AAC BrainControl Interface device, which uses an EEG helmet to detect users’ brain activity — to help them interact with the outside world. By leveraging machine learning techniques for language generation, particularly using recurrent networks and large language models, it is possible to accurately predict user intent and improve their typing experience. Experimental results show the average number of interactions reduced by a factor of 2.66 compared to the original sequential key scanning method, which is extremely significant for locked-in patients.