Self-Enhanced Feature Fusion for RGB-D Semantic Segmentation

Abstract

Effectively fusing depth and RGB information to fully leverage their complementary strengths is essential for advancing RGB-D semantic segmentation. However, when fusing with RGB information, traditional methods often overlook noises in depth data, presuming that they are of high accuracy. To resolve this issue, we propose a self-enhanced feature fusion network (SEFnet) for RGB-D semantic segmentation in this work. It mainly comprises three steps. Firstly, RGB and depth embeddings from the initial layers of the network are fused together. Secondly, the fused features are enhanced by pure RGB embeddings and are progressively guided by semantic edge labels to suppress irrelevant features. Finally, the enhanced features are combined with high-level RGB features and are fed into a normalizing flow decoder to obtain segmentation results. Experimental results demonstrate that the proposed approach can provide accurate predictions, outperforming state-of-the-art methods on benchmark datasets.