CPIFuse: Toward realistic color and enhanced textures in color polarization image fusion

Abstract

Conventional image fusion aims to integrate multiple sets of source images into one with more details, representing the merging of intensity information. In contrast, polarization image fusion seeks to enhance texture of the intensity image $S_0$ in the corresponding spectral bands by integrating strong texture features reflected by DoLP (Degree of Linear Polarization) images, representing the combination of intensity and polarization, which are both physical properties of light. However, the 3-Dimensional information contained in DoLP is presented in a highlighted form within the 2-Dimensional image, and fusing it directly can result in spectrum discontinuities and obscuring necessary details of the fused image. Existing polarization image fusion methods do not analyze this phenomenon and fail to examine the physical information represented by DoLP images. Instead, they simply integrate this interference information in the same manner as fusing infrared images, leading to fused images that suffer from information loss and significant color discrepancies. In this paper, we propose a new color polarization image fusion strategy that takes into account the physical properties reflected in the $S_0$ and DoLP images, namely CPIFuse. CPIFuse designs a customized loss function to optimize parameters through a lightweight Transformer-based image fusion framework, and color polarization image fusion has been achieved with color fidelity, enhanced texture and high efficiency. These advantages can be demonstrated in the visual effects, quantitative metrics, and car detection tasks of our comparative experiments. Furthermore, a new polarization dataset is constructed by the mechanism of division of focal plane polarimeter camera, which addresses the scarcity of datasets in the field of polarization image fusion. The source code and CPIF-dataset will be available at https://github.com/roydon-luo/CPIFuse.