Real-Time Siamese Visual Tracking with Lightweight Transformer

Abstract

Trackers based on Siamese have demonstrated more remarkable performance in visual tracking. The majority of existing trackers typically compute target template and search image features independently, then utilize cross-correlation to predict the possibility of an object appearing at each spatial position in the search image for target localization. This paper proposes a Siamese network for feature enhancement and aggregation between the target template and the search image by utilizing a lightweight transformer with several linear self- and cross-attention layers. With anchor-free head prediction, the suggested framework is simple and effective. Extensive experiments on visual tracking benchmarks such as VOT2018, UAV123, and OTB100 demonstrates that our tracker achieves state-of-the-art performance and operates at a real-time frame rate of 39 fps.