Reconstructing Visual Stimulus Representation From EEG Signals Based on Deep Visual Representation Model

Abstract

Reconstructing visual stimulus representation is a significant task in neural decoding. Until now, most studies have considered functional magnetic resonance imaging (fMRI) as the signal source. However, fMRI-based image reconstruction methods are challenging to apply widely due to the complexity and high cost of acquisition equipment. Taking into account the advantages of the low cost and easy portability of electroencephalogram (EEG) acquisition equipment, we propose a novel image reconstruction method based on EEG signals in this article. First, to meet the high recognizability of visual stimulus images in a fast-switching manner, we construct a visual stimuli image dataset and obtain the corresponding EEG dataset through EEG signals collection experiment. Second, we introduce the deep visual representation model (DVRM), comprising a primary encoder and a subordinate decoder, to reconstruct visual stimuli representation. The encoder is designed based on residual-in-residual dense blocks to learn the distribution characteristics between EEG signals and visual stimulus images. Meanwhile, the decoder is designed using a deep neural network to reconstruct the visual stimulus representation from the learned deep visual representation. The DVRM can accommodate the deep and multiview visual features of the human natural state, resulting in more precise reconstructed images. Finally, we evaluate the DVRM based on the quality of the generated images using our EEG dataset. The results demonstrate that the DVRM exhibits an excellent performance in learning deep visual representation from EEG signals, generating reconstructed representation of images that are realistic and highly resemble the original images.