Attention Guided Filter and Refinement Feature Network for image semantic segmentation

Abstract

Global information and spatial texture are fundamental for optimizing the performance of segmentation networks. Although atrous convolution effectively enlarges receptive fields to accommodate multi-scale features, it cannot capture directional pixel correlations adequately. Moreover, fusing features from different levels via summation or concatenation can introduce substantial noise, compromising segmentation quality. To address these issues, we developed the Attention-Guided Filtering and Refinement Feature Network (FRFN), which enhances global information representation in deeper layers while minimizing noise in shallow features. The Dense Pyramid Attention Module (DPAM) embedded within FRFN captures multi-scale, long-range contextual dependencies. Additionally, the Strip Compression Spatial Block (SCSB) integrated into DPAM extends the long-range pixel interactions through strip convolution. The Enhancement Fusion Module (EFM) also filters noise in shallow features, enhancing the capacity to capture global information. Extensive experiments on the PASCAL VOC 2012 and Cityscapes test datasets, as well as the COCO-Stuff-164K validation set, validate the efficacy of our proposed methods, with FRFN achieving 83.5% and 80.1% $m$IoU on the respective test datasets, and 40.18% $m$IoU on the COCO-Stuff-164K validation set.