RGB-T object tracking with adaptive decision fusion

Visual object tracking is a traditional task in computer vision, which has developed with several decades. With the development of machine learning, Correlation Filter (CF) has been proposed with satisfying performance and very high framerate. Though the CF framework has numerous strengths in this task, the tracker is fragile to miss the target in several scenes, including extreme illumination, target occlusion and deformation. Recently, thermal modality, which detects the target’s temperature, is robust to the night scenes and can provide a precise target contour. In this paper, we propose a CF based tracker with decision fusion strategy for visible-thermal (RGB-T) tracking. First, we introduce multi-modal KCF trackers as our baseline. Then, we design a decision fusion method considering the Peak-to-Side Rate (PSR) of the score maps, thereby achieving an adaptive fusing those modalities and avoiding model’s heterogeneity. In the experiments, our tracker has validated on the public dataset, namely GTOT. Compared with two uni-modality trackers, the proposed tracker with real-time speed has shown superior results on both target localization and scale estimation.