OSFusion: A One-Stream Infrared and Visible Image Fusion Framework

Abstract

The current popular two-stream two-stage image fusion framework extracts features of infrared and visible images separately and then performs feature fusion. The extracted features lack interaction between the source images and have limited cross-modal complementary capability. To address these issues, we propose a novel one-stream infrared and visible image fusion (OSFusion) framework that connects a source image pair to achieve bidirectional information flow. In this way, the fused features with cross-modal complementary information can be dynamically extracted by mutual guidance. To further improve the inference efficiency and obtain high-quality fused images, a feature extraction and fusion module (FEFM) is proposed based on Transformer structure. The combination of feature extraction and feature fusion is realized by using it. Since there is no need for an extra feature interaction module and the implementation is highly parallel, the speed of image fusion is extremely fast. Benefiting from the one-stream structure and FEFM, OSFusion achieves promising infrared and visible image fusion performance on MSRS, M3FD, and RoadScene datasets. Besides, our method achieves a good balance in the trade-off between performance and complexity, and also shows a faster convergence trend.