TLSH-MOT: Drone-View Video Multiple Object Tracking via Transformer-Based Locally Sensitive Hash

Abstract

Multiple object tracking (MOT) plays an essential role in drone-view remote sensing applications, such as urban management, emergency rescue, and maritime monitoring. However, due to large variations in object scale and position, the frequent feature loss across frames, and difficulties in matching, traditional methods struggle to achieve high-tracking accuracy in such challenging environments. To address these issues, we propose a Transformer-based locally sensitive hash MOT (TLSH-MOT) method in drone-view remote sensing scenarios. First, a frame-level feature extraction and enhancement module is introduced, integrating a nominee proposal generation (NPG) unit and a tilt convolutional vision Transformer (ViT), which enables adaptive detection of objects across varying scales and perspectives. Next, a spatiotemporal memory (STM) structure is designed to mitigate instantaneous environmental interference and periodic changes using short-term and long-term memory blocks, thereby enhancing tracking stability under complex meteorological conditions. In addition, a temporal enhancement feature decoder (TEFD) fuses multisource feature information to better understand the motion patterns of remote sensing objects. Finally, a local sensitive hash (LSH) IDLinker ensures efficient feature matching, significantly improving trajectory association in large-scale monitoring scenarios. Experimental results show that TLSH-MOT achieves MOT accuracy of 40.7% and 62.2% on VisDrone and UAVDT datasets, respectively, which verifies the superiority of TLSH-MOT in the remote sensing video tracking field. The framework’s code is released at: https://github.com/YubinYuan/TLSH-MOT.