MCAFNet: Multiscale Channel Attention Fusion Network for Arbitrary Style Transfer

Abstract

Recently, attention-based arbitrary style transfer (AST) techniques have been widely applied in image generation and video processing. However, the scale bias of the attention module used for contextual information extraction and multiscale feature aggregation poses a challenge in balancing the content structure and style patterns of images. In this work, a multiscale channel attention fusion network (MCAFNet) is proposed to generate stylization images with well-coordinated content and style. Specifically, the multiscale channel attention module (MCAM) is introduced to extract both local and global contextual information of style features within the channel dimension and subsequently aggregate this information with content features. Following MCAM, an attentional feature fusion module (AFFM) is adopted to effectively integrate both deep and shallow semantic features. Furthermore, a novel contrastive loss based on multi-source feature enhancement is proposed to optimize the spatial distribution between content and style features. Both qualitative and quantitative experimental results compared to the state-of-the-art (SOTA) baseline approaches indicate the superiority of the proposed method for real-time image and video style transfer.