ESCS: An Expandable Semantic Communication System for Multimodal Data Based on Contrastive Learning

Abstract

Semantic communication represents a novel paradigm for meeting the substantial data transmission demands of sixth-generation (6G) mobile communications. To address the challenges of information redundancy, conflicts, asynchrony, and internal interference among multimodal data, this letter introduces an expandable semantic communication system (ESCS). We propose a generic multimodal cross-attention (MMCA) module that enhances interactions among heterogeneous features under the guidance of an autonomously selected leader. By employing dual-contrastive learning, we impose stringent requirements for the feature representation capabilities of the transmitter, enabling it to differentiate between samples containing heterogeneous data. Evaluation results from four tasks under additive white Gaussian noise (AWGN) and fading channels indicate that the proposed system significantly outperforms state-of-the-art methods regarding storage overhead, task accuracy, and channel utilization.