PI net: An end-to-end semantic decoding model for EEG signals in perception and imagination tasks

Abstract

Existing methods for decoding the semantics of perception and imagination based on EEG signals primarily focus on isolated tasks or single modalities, neglecting the similarities between perception and imagination EEG signals. This leads to significant limitations in performance and applicability when data types are limited or lack diversity. To address these issues, this paper proposes a novel model, PI Net, for decoding perception and imagination activities. PI Net fully utilizes the similarities between perception and imagination EEG signals, placing them in the same framework for classification, thereby improving the model's decoding performance and generalizability. PI Net first extracts channel and temporal features from EEG signals as shallow features. Then, the Dynamic Convolution in the Spatial-Temporal-Frequency Domain module dynamically adjusts convolution weights based on the spatial-temporal-frequency domain characteristics of the input EEG signals, enabling adaptive processing of temporal-spatial-spectral features. The Assign Feature Weights module, based on the GAU linear attention mechanism, adaptively increases the weights of important features. Finally, PI Net outputs the prediction results for different categories under various tasks through Fully Connected Classification Output using fully connected layers and Softmax layers. Experimental results on publicly available multimodal perception and imagination datasets show that PI Net achieves an accuracy of 92.85% for binary classification and 19.89% for complex 18-class classification tasks, outperforming other models and demonstrating its superior decoding performance and generalizability.