{"title":"SARFormer: Segmenting Anything Guided Transformer for semantic segmentation","authors":"Lixin Zhang , Wenteng Huang , Bin Fan","doi":"10.1016/j.neucom.2025.129915","DOIUrl":null,"url":null,"abstract":"<div><div>Semantic segmentation plays a crucial role in robotic systems. Despite advances, we find that current state-of-the-art methods are hard to apply in practice due to their weak generalization ability. Especially, diffusion-based segmentation methods struggle with over-reliance on noisy Ground Truth (GT) annotations, which are corrupted with noise and directly fed into the model’s forward propagation process during training, limiting the model’s ability to generalize. While the Segment Anything Model (SAM) excels at instance segmentation, it faces challenges in controlling granularity and lacks semantic information. To address these issues, we propose SARFormer, a semantic segmentation algorithm guided by SAM. Unlike conventional methods, SARFormer uses GT solely for supervision and replaces noisy GT with SAM guidance, enabling better generalization. The key innovations include a region-based SAM optimizer to refine granularity and a feature aggregation method for enhanced deep feature extraction. Experimental results show SARFormer achieves competitive accuracy, demonstrating the effectiveness of SAM in improving segmentation performance</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"635 ","pages":"Article 129915"},"PeriodicalIF":5.5000,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurocomputing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0925231225005879","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Semantic segmentation plays a crucial role in robotic systems. Despite advances, we find that current state-of-the-art methods are hard to apply in practice due to their weak generalization ability. Especially, diffusion-based segmentation methods struggle with over-reliance on noisy Ground Truth (GT) annotations, which are corrupted with noise and directly fed into the model’s forward propagation process during training, limiting the model’s ability to generalize. While the Segment Anything Model (SAM) excels at instance segmentation, it faces challenges in controlling granularity and lacks semantic information. To address these issues, we propose SARFormer, a semantic segmentation algorithm guided by SAM. Unlike conventional methods, SARFormer uses GT solely for supervision and replaces noisy GT with SAM guidance, enabling better generalization. The key innovations include a region-based SAM optimizer to refine granularity and a feature aggregation method for enhanced deep feature extraction. Experimental results show SARFormer achieves competitive accuracy, demonstrating the effectiveness of SAM in improving segmentation performance
期刊介绍:
Neurocomputing publishes articles describing recent fundamental contributions in the field of neurocomputing. Neurocomputing theory, practice and applications are the essential topics being covered.