SCDFuse: A semantic complementary distillation framework for joint infrared and visible image fusion and denoising

Abstract

Infrared and visible fusion has gained unprecedented attention due to its extensive applications in the field of computer vision. However, existing algorithms unilaterally focus on the fusion of clean scene images and are vulnerable to noise interference. Although this issue can be mitigated by deploying independent pre-denoising modules, the cascading of additional modules with diverse functionalities introduces supplementary complexity, computational overhead, and even inter-module interference. To overcome this limitation and achieve multitask unification, we propose a knowledge distillation framework for end-to-end simultaneous feature denoising and aggregation. In this framework, we leverage the advantages of the distillation architecture to generate soft labels, mitigating unstable fusion performance caused by lacking of label guidance. To achieve an accurate guidance for the function learning during the training process, an asymmetric noise-aware training strategy is devised for the facilitate of aggregation robustness and denoising capability. Moreover, to ensure the feature excavation and semantic complementary competence, a hybrid series–parallel CNN-transformer dual-branch En-Decoder is constructed. The proposed encoder incorporate the self-designed Textural-aware ConvNextV2, strip pooling attention and progressive residual transformer to compose the dual-branch architecture. In addition, the semantic complementary feature aggregation (SCFA) module are developed to realize a coarse-to-fine feature enhancement. Extensive experiments on both regular and noisy fusion materials are implemented to testify the integration and denoising performance of the proposed method. Notably, on the TNO dataset, the proposed method achieves improvements of 4% and 4.2% in the MSSIM and UQI metrics, respectively, compared to the second-best algorithm. Furthermore, we also investigate its facilitation for advanced visual tasks through object detection experiments.