Attention Guided Class Activation Maps for Boosting Weakly Supervised Semantic Segmentation

Abstract

Weakly Supervised Semantic Segmentation (WSSS) has garnered significant attention for its ability to utilize weaker labels in place of expensive pixel-level annotations while maintaining commendable performance. Class Activation Maps (CAM) can still possess target localization capabilities without pixel-level annotations, and are thus widely used as pseudo labels to supervise subsequent segmentation tasks. With the continuous advancement of artificial intelligence, generating high-quality CAM by combining the strengths of Convolutional Neural Networks (CNN) and Transformer architectures has received widespread attention. As a classic architecture, Conformer adopts a parallel structure and has been applied in multiple WSSS models. We observed that the attention matrices of different levels of Transformer blocks in Conformer exhibit significant characteristic differences, and these matrices effectively capture the correlations between different regions. Based on this observation, we propose a method called Attention-Guided Class Activation Map (AG-CAM), which selectively utilizes attention matrices at different levels to enhance features for various purposes. Detailed experiments on common datasets have shown that the proposed AG-CAM method significantly improves the quality of class activation maps. Our work provides a more precise solution for WSSS, thereby demonstrating immense potential and value in real-world applications where data annotation is scarce.