Automatic calibration and registration of lidar and stereo camera without calibration objects

Perception of the environment is an important task for intelligent vehicles, and to effectively perceive the environment, multiple sensors are often employed. In this paper, we propose to integrate the perceived data from 3D LIDAR and stereo camera using particle swarm optimization algorithm, without the aid of any external calibration aids. The proposed optimisation algorithm automatically calibrates and registers the LIDAR range image and stereo depth image, as a precursor to the sensor fusion. Multiple parameters are optimised by adopting a model-based approach during the parameter estimation phase. The evaluation of the parameters is performed using a novel depth-based cost function. During the sensor fusion phase, the optimised parameters are used to generate the LIDAR range image, which functions as the disparity range image for the Viterbi-based stereo disparity estimation. The disparity range image constrains the Viterbi search during the stereo disparity estimation. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.