Camera and LiDAR Sensor Fusion for 3D Object Tracking in a Collision Avoidance System

Abstract

Advances in autonomous sensor technology are the driving force behind vehicle manufacturers to reduce traffic accidents and fatalities. This process led to the development of advanced driver assistant systems (ADAS). However, vision-based methods often suffer from limited fields of view and difficulty to extract accurate range information which is critical for vehicle detection. Vehicle detection is one of the most important issues for ADAS. The authors show one among many other possible solutions, to improve detection, and that is to rely on several different sensors such as a camera and LiDAR sensors. This paper describes the implementation of a collision-avoidance system (CSA), and the needed time-to-collision (TTC) estimation, using constant velocity model and C++ programming language. The TTC calculation is based on data obtained by tightly coupled LiDAR and camera sensors. In the solution we integrated several key points detectors and, focused on descriptors extraction and matching. As a final step, we used so called, sensor fusion to integrate LiDAR points into camera images and detect object in camera images using deep learning approach. For evaluation, we run several combinations of detectors and descriptors, analyze differences between time to collision estimations and demonstrate the functionality of the best method into an efficient implementation.