FHFN: content and context feature hierarchical fusion networks for multi-focus image fusion

Abstract

Thanks to many current deep learning-based multi-focus image fusion methods have the defects of over-extracting image local features or neglecting image global features, these methods lead to final fused images with color distortion, small-area artifacts, large-area blurring, and unsoft boundary transitions. To solve these problems, we propose a new global and local feature hierarchical fusion network for multi-focus image fusion, called FHFN. The proposed FHFN is a deep neural network that simultaneously extracts global features using Swin Transformer and local features using ConvNeXt. On the one hand, we use the PSA module to enhance the focus on local features of images and effectively interact shallow features and high-level semantic features. On the other hand, we design the hierarchical fusion of extracted local features and global features by the hierarchical feature fusion module (HFFB), which constitutes a new image fusion task paradigm for solving multi-focus image fusion tasks. On the other hand, we introduce the gradient residual dense module (RGDB) to strengthen the edge features of images and improve the extraction capability of fine-grained spatial features of the network. Our method is competitive with ten other MFIF methods on four public datasets in terms of both objective quantitative metrics and subjective visual perception, and outperforms other MFIF methods in the same field.