Pedestrian fusion tracking method based on multimodal information com-plementation

Abstract

A pedestrian fusion tracking method with complementary multimodal information is proposed, and a fusion decision tracking model with detection followed by fusion and then tracking is established. The detection module uses a modified CenterNet network with a richer feature information backbone network and a lightweight prediction module, and the scene data is collected to train multiple detectors for multiple modalities. A decision process based on the confidence of detection results and feature similarity is proposed to achieve the fusion of multimodal detection results, and the fused results are fed to the tracker to achieve continuous pedestrian tracking. The results show that the proposed fusion tracking model can complement each other’s multi-modal information and provide better and more robust tracking results than the single-modal tracker for continuous tracking in multiple scenes.