Multilevel Weighted Feature Fusion Using Convolutional Neural Networks for EEG Motor Imagery Classification

Abstract

Deep learning methods, such as convolution neural networks (CNNs), have achieved remarkable success in computer vision tasks. Hence, an increasing trend in using deep learning for electroencephalograph (EEG) analysis is evident. Extracting relevant information from CNN features is one of the key reasons behind the success of the CNN-based deep learning models. Some CNN models use convolutional features from different CNN layers with good effect. However, extraction and fusion of multilevel convolutional features remain unexplored for EEG applications. Moreover, cognitive computing and artificial intelligence experience increasing applications in all fields. Cognitive process is based on understanding human brain cognition through signals, such as EEG. Hence, deep learning can aid in developing cognitive systems and related applications by improving EEG decoding. The classification and recognition of EEG have consistently been challenging due to its characteristics of dynamic time series data and low signal-to-noise ratio. However, the information hidden in different convolution layers can aid in improving feature discrimination capability. In this paper, we use the EEG motor imagery data to uncover the benefits of extracting and fusing multilevel convolutional features from different CNN layers, which are abstract representations of the input at various levels. Our proposed CNN model can learn robust spectral and temporal features from the raw EEG data. We demonstrate that such multilevel feature fusion outperforms the models that use features only from the last layer. Our results are better than the state of the art for EEG decoding and classification.