Learning Motor Cues in Brain-Muscle Modulation

Abstract

Current studies for brain-muscle modulation often analyze selected properties in electrophysiological signals, leading to a partial understanding. This article proposes a cross-modal generative model that converts brain activities measured by electroencephalography (EEG) to corresponding muscular responses recorded by electromyography (EMG). Examining the generation process in the model highlights how the motor cue, representing implicit motor information hidden within brain activities, modulates the interaction between brain and muscle systems. The proposed model employs a two-stage generation process to bridge the semantic gap in cross-modal signals. Initially, the shared movement-related information between EEG and EMG signals is extracted using a contrastive learning framework. These shared representations act as conditional vectors in the subsequent EMG generation stage based on generative adversarial networks (GANs). Experiments on a self-collected multimodal electrophysiological signal data set show the algorithm’s superiority over existing time series generative methods in cross-modal EMG generation. Further insights derived from the model’s inference process underscore the brain’s strategy for muscle control during movements. This research provides a data-driven approach for the neuroscience community, offering a comprehensive perspective of brain-muscular modulation.