Encoding context and decoding aggregated information for semantic segmentation

Abstract

In the process of feature extraction, the existing encoder–decoder network models often use the continuous downsampling method to expand the receptive field, which directly leads to the reduction of the model’s ability to obtain fine-grained spatial information, and further makes it difficult to recover the lost spatial details in the process of upsampling the feature map. In addition, the direct fusion of the features of the encoder and the decoder leads to the problem that the detailed features are masked by the semantic features. To address these challenges, we build a new semantic segmentation model named ECDAISeg. The model adopts an encoder–decoder structure. We embed Context Propagation Module (CPM) and Blend Feature Balance Module (BFBM) between the encoder and the decoder. The role of CMP is to recover lost detail information after feature extraction and to provide multi-scale contextual information for better understanding of global semantics. The BFBM is used to balance high-level semantic information with low-level detailed information through the attention mechanism, thereby filtering out redundant information and preserving important details. Evaluations on the PASCAL VOC 2012 and Cityscapes validation sets conclude that ECDAISeg achieves 82.85% and 74.49% mIoU, realizing better segmentation results compared to various representative segmentation models.