Multi-Object Tracking with Spatial-Temporal Correlation Memory Networks

Resistance to object appearance deformation and local occlusion is still one of the challenges of multi-object tracking algorithms. Most popular algorithms rely on time-consuming numerical optimization and complex manual design strategies to integrate object appearance information and motion information, so as to alleviate the adverse effects of object appearance deformation and local occlusion on the trajectory updating. This paper proposes a Spatial-Temporal Correlation Memory (STCM) module which can adaptively aggregate useful information from rich historical information in memory. By mining the time dimension information, the STCM module can guide the backbone network to extract the current frame effectively, and adapt to the change in the object’s appearance in the tracking process. Specifically, the STCM module can record the foreground-background information in the history frames and direct the backbone network to focus on the useful information in the current frame. Experiments on the MOT17 data set show that our method outperforms the baseline method and current advanced method in index MOTA and IDFI.