Siam2C: Siamese visual segmentation and tracking with classification-rank loss and classification-aware

Siamese visual trackers based on segmentation have garnered considerable attention due to their high accuracy. However, these trackers rely solely on simple classification confidence to distinguish between positive and negative samples (foreground or background), lacking more precise discrimination capabilities for objects. Moreover, the backbone network excels at focusing on local information during feature extraction, failing to capture the long-distance contextual semantics crucial for classification. Consequently, these trackers are highly susceptible to interference during actual tracking, leading to erroneous object segmentation and subsequent tracking failures, thereby compromising robustness. For this purpose, we propose a Siamese visual segmentation and tracking network with classification-rank loss and classification-aware (Siam2C). We design a classification-rank loss (CRL) algorithm to enlarge the margin between positive and negative samples, ensuring that positive samples are ranked higher than negative ones. This optimization enhances the network’s ability to learn from positive and negative samples, allowing the tracker to accurately select the object for segmentation and tracking rather than being misled by interfering targets. Additionally, we design a classification-aware attention module (CAM), which employs spatial and channel self-attention mechanisms to capture long-distance dependencies between different positions in the feature map. The module enhances the feature representation capability of the backbone network, providing richer global contextual semantic information for the tracking network’s classification decisions. Extensive experiments on the VOT2016, VOT2018, VOT2019, OTB100, UAV123, GOT-10k, DAVIS2016, and DAVIS2017 datasets demonstrate the outstanding performance of Siam2C.