A New Approach to Lidar and Camera Fusion for Autonomous Driving

Abstract

In this paper, we introduce an object detection model that combines a camera and a LiDAR sensor. In previous object detection studies have mainly focused on using one sensor, and mainly camera and LiDAR sensors were used. Research was mainly conducted in the direction of utilizing a single sensor, and typically cameras and LiDAR sensors were used. However, Camera and Li-DAR sensors have disadvantages such as being vulnerable to environmental changes or having sparse expressive power, so the method to improve them is needed for a stable cognitive system. In this paper, we propose the LiDAR Camera Fusion Network, a sensor fusion object detection model that uses the advantages of each sensor to improve the disadvantages of cameras and Li-DAR sensors. The sensor fusion object detector developed in this study has the feature of estimating the location of an object through LiDAR Clustering. Extraction speed is about 58 times faster than Selective search without prior learning, reducing the number of candidate regions from 2000 to 98, despite reducing the number of candidate regions, compared to existing methods, the ratio of the correct answer candidate areas among the total location candidate regions was 10 times larger. Due to the above characteristics, efficient learning and inference were possible compared to the existing method, and this model finally extracts the probability value of the object, the bounding box correction value, and the distance value from the object. Due to the characteristic of our research, we used KITTI data because LiDAR and image data were needed. As a result, we compare the results with object detection models that are often used in the object detection area.