Extrinsic Calibration of LiDAR-Camera Based on Deep Convolutional Network

Abstract

LiDAR and stereo cameras are increasingly being used for intelligent perceptual tasks in autonomous vehicles and robotic platforms. However, before the sensors can be used, they usually need to be precisely calibrated both internally and externally considering the calibration affection of the sensor parameters. With the increasing popularity of deep learning (DL), some recent studies have proved the advantages of DL in feature extraction, feature matching and global regression in extrinsic calibration. To improve the accuracy and reduce calibration time, we propose a method for automatic extrinsic calibration of LiDAR and stereo camera based on deep convolutional network. It has the nonlinear mapping ability of neural network to establish the mapping relationship between the target in the LiDAR coordinate system and its image pixel coordinate system. Moreover, the proposed method does not require the resort to any extra calibrator, which reduces some manual steps and compensates some shortcomings of traditional methods. The method can be used for the extrinsic calibration of LiDAR and camera online, which is meaningful for further fusing the sensor data.