FIAFusion: A Feedback-Based Illumination-Adaptive Infrared and Visible Image Fusion Method

Abstract

Infrared (IR) and visible (VI) image fusion enables to combine the strengths of both original images adequately, retaining essential target information and abundant detailed textures. Existing fusion methods mainly cater to well-illuminated scenes. Although some researchers have explored complex scenes, there are still some unresolved issues, such as suboptimal lighting levels and loss of local details. To overcome these issues, we introduce a novel method named FIAFusion. FIAFusion is structured into three primary components: initially, the illumination-adaptive network (IAN) adjusts the illumination of the original VI image adaptively. Subsequently, the fusion network (FUN) efficiently merges the complementary information from the original IR image and the illumination-adapted VI image into a fused image of high visual quality. To achieve an ideal illumination level in the fused image, the feedback network (FEN) is designed to feedback on the illumination information of the fused image to both IAN and FUN, guiding the illumination correction to facilitate mutual promotion between illumination adaptation and fusion process effectively. Extensive comparative and supplementary experiments conducted on LLVIP and MSRS datasets indicate that our method surpasses state-of-the-art (SOTA) IR and VI image fusion methods. Moreover, our method demonstrates significant performance improvements in pedestrian detection tasks.