Content Adaptive Distributed Joint Source-Channel Coding for Image Transmission With Hyperprior

Abstract

We study deep joint source-channel coding (JSCC) in a distributed dual-view scenario where the sources are correlated images and the channels are independent. The transmitters encode respective images and then send them to the same central receiver. In this situation, how to effectively use the correlation between sources to enhance the reconstruction quality is worth investigating. In existing deep JSCC studies, the information fusion strategy failed to effectively utilize the correlation between images from different perspectives at the receiver. In this paper, we propose an information fusion module based on multi-layer cross-attention mechanism to fuse image features at different pixel levels to make full use of the source correlation. In addition, while most previous studies allocated the same bandwidth to all images, which ignored the differences between images, we design a content adaptive variable-rate module based on the proposed entropy mask. We conduct experiments on KITTI and InStereo2K datasets and evaluate them using peak signal-to-noise ratio (PSNR) and multi-scale structural similarity index (MS-SSIM) metrics. The experimental results show that our proposed multi-layer information fusion module and entropy mask module can effectively improve the quality of reconstruction by about 1.0 dB PSNR compared to the state-of-the-art distributed JSCC.