A dual channel-cross fusion network for polarization image fusion

Abstract

Polarization image fusion utilizes images with different polarization directions to generate fused images containing rich texture information and intensity information. Typically, the addition or concatenation operations are usually used in fusion methods, while these approaches are ineffective in merging the features from different source images. Therefore, a dual channel-cross fusion network for polarization image fusion is proposed, termed as DCCFNet. The network has two main modules: feature extraction & fusion module and generation module. Feature extraction & fusion module has two main branches, which are used to extract the features of the polarization images. In each branch, a squeeze-and-excitation based (SE-based) block is used to obtain the global information distribution of the feature maps within the channel dimension. To better fuse the features from different source images, a cross-fusion branch is proposed to connect the two branches and interleave the feature maps across the channel dimension. Also, a spatial attention (SA) block is employed to capture the key regions of the images. Generation module is used to generate the fused images from features. In addition, a loss function that integrates weighted structural similarity index measure (MSWSSIM) loss, intensity loss and gradient loss is developed to preserve more structural and intensity information from the source images and enhancing the clarity of target edges. The results of the experiments on two public datasets have proved the effectiveness and advancement of our proposed method.