Calibrank: Effective Lidar-Camera Extrinsic Calibration By Multi-Modal Learning To Rank

Precise and online LiDAR-camera extrinsic calibration is one of the prerequisites of multi-modal data fusion for autonomous perception. The existing 6-DoF pose regression networks take majority effort on coarse-to-fine training strategy to gradually approach the global minimum. However, with limited computing resources, the optimal pose parameters seem unreachable. Moreover, recent research on neural network interpretability reveals that learning-based pose regression is nothing but the interpolation with most relevant samples. Motivated by this notion, we propose to solve the calibration problem in a retrieval way. Concretely, the learning-to-rank pipeline is introduced for ranking the top n relevant poses in the gallery set, which is then fused in to the final prediction. To better explore the pose relevance between ground truth samples, we further propose an exponential mapping from parametric space to the relevance space. The superiority of the proposed method is validated and demonstrated in the comparative and ablative experimental analysis.