TS-BiT: Two-Stage Binary Transformer for ORSI Salient Object Detection

Abstract

Vision transformers (ViTs) have demonstrated superior performance in various remote sensing tasks, such as optical remote sensing image salient object detection (ORSI-SOD). However, the high resolution of remote sensing images and the substantial computational costs pose significant challenges for deploying existing methods on resource-constrained devices. Model binarization significantly reduces computational costs and storage requirements by constraining weights and activations to 1-bit representations, which has been widely explored in convolutional neural networks (CNNs). However, directly applying binary methods to ViTs poses challenges since quantization errors hinder the ability to capture the similarity between tokens, resulting in significant performance degradation in detecting salient objects in complex ORSI scenarios. To address this issue, we propose two-stage binary transformer (TS-BiT) for the ORSI-SOD task to preserve information on salient objects under 1-bit representation. Specifically, we design a two-stage central-aware softmax binarization (TCSB) strategy to reduce quantization errors arising from substantial discrepancies in the long-tail distribution of multihead attention. Furthermore, we develop a scalable hyperbolic tangent function to approximate the gradients of the Sign function within each binarization group, substantially mitigating quantization errors during the binarization of softmax attention. Extensive experiments demonstrate that our method outperforms existing binary ViT approaches on ORSSD, EORSSD, and ORSI-4199 datasets.