Multi-Task Occlusion Learning for Real-Time Visual Object Tracking

Abstract

Occlusion handling is one of the important challenges in the field of visual tracking, especially for real-time applications, where further processing for occlusion reasoning may not always be possible. In this paper, an occlusion-aware real-time object tracker is proposed, which enhances the baseline SiamRPN model with an additional branch that directly predicts the occlusion level of the object. Experimental results on GOT-10k and VOT benchmarks show that learning to predict occlusion levels end-to-end in this multi-task learning framework helps improve tracking accuracy, especially on frames that contain occlusions. Up to 7% improvement on EAO scores can be observed for occluded frames, which are only 11% of the data. The performance results over all frames also indicate the model does favorably compared to the other trackers.