Real-Time RGB-D Pedestrian Tracking for Mobile Robot

Pedestrian tracking is an important research direction in the field of mobile robotics. In order to complete tasks more efficiently and without hindering the original intention of pedestrians, mobile robots need to track pedestrians accurately in real time. In this paper, we propose a real-time RGB-D pedestrian tracking framework. First, we propose a pedestrian segmentation detection algorithm to detect pedestrians and obtain their two-dimensional positions. Second, due to limited computational resources and the rarity of missed detection for pedestrians, we use an nearest neighbor tracker for pedestrian tracking. To address the issue of inaccurate pedestrian localization, we use our detection algorithm to obtain the center of pedestrians from RGB images. By combining them with point clouds, the 2D coordinates of pedestrians are obtained. Our method enables accurate pedestrian tracking in the world coordinate, by adaptively fusing RGB images with their corresponding depth-based point clouds. Besides, our light-weight detection and tracking algorithm guarantee the real-time pedestrian tracking for realistic mobile robot applications. To validate the effectiveness and real-time performance of tracking algorithm, we conduct experiments using multiple pedestrian datasets of approximately half a minute in length, captured from two different perspectives. To validate the practicality and accuracy of the tracking algorithm in real-world scenarios, we extend our tracking algorithm to apply it to trajectory prediction.