Improving auditory attention decoding by classifying intracranial responses to glimpsed and masked acoustic events

Abstract Listeners with hearing loss have trouble following a conversation in multitalker environments. While modern hearing aids can generally amplify speech, these devices are unable to tune into a target speaker without first knowing to which speaker a user aims to attend. Brain-controlled hearing aids have been proposed using auditory attention decoding (AAD) methods, but current methods use the same model to compare the speech stimulus and neural response, regardless of the dynamic overlap between talkers which is known to influence neural encoding. Here, we propose a novel framework that directly classifies event-related potentials (ERPs) evoked by glimpsed and masked acoustic events to determine whether the source of the event was attended. We present a system that identifies auditory events using the local maxima in the envelope rate of change, assesses the temporal masking of auditory events relative to competing speakers, and utilizes masking-specific ERP classifiers to determine if the source of the event was attended. Using intracranial electrophysiological recordings, we showed that high gamma ERPs from recording sites in auditory cortex can effectively decode the attention of subjects. This method of AAD provides higher accuracy, shorter switch times, and more stable decoding results compared with traditional correlational methods, permitting the quick and accurate detection of changes in a listener’s attentional focus. This framework also holds unique potential for detecting instances of divided attention and inattention. Overall, we extend the scope of AAD algorithms by introducing the first linear, direct-classification method for determining a listener’s attentional focus that leverages the latest research in multitalker speech perception. This work represents another step toward informing the development of effective and intuitive brain-controlled hearing assistive devices.