Monocular 3D Multi-Object Tracking with an EKF Approach for Long-Term Stable Tracks

Abstract

Most monocular object tracking algorithms work in the 2D domain of the image. However, object trajectories, which are very simple in a fixed 3D world space, result in complex motions on the image plane, especially when the camera is moving. Therefore, in absence of any 3D representation, aforementioned approaches are only able to perform the measurement-to-track association based on rough similarity of 2D bounding box parameters. Recent advances in monocular 3D object detection allow to extract additional parameters like the pose and spatial extent of a 3D bounding box. In this paper, we present a multi-object tracking approach composed of an Extended Kalman filter estimating the 3D state by using these detections for track initialization. In subsequent time steps 2D bounding boxes are used to avoid filtering temporally correlated 3D measurements. This ensures properly estimated state uncertainties. We show that this 3D representation is very valuable as we achieve state-of-the-art results on the KITTI dataset with an association solely based on 2D bounding box comparison. We use state uncertainties transformed into the measurement space while completely ignoring appearance features.