Camera-LiDAR Fusion With Latent Correlation for Cross-Scene Place Recognition

Abstract

As a crucial issue for outdoor mobile robot navigation, place recognition (PR) remains challenging in long-term cross-scene applications, necessitating enhanced robustness of PR algorithms. In this article, a novel multimodal PR approach is proposed, which deeply fuses the camera and LiDAR to effectively compensate for their respective limitations. The introduced place descriptor consists of three branches: image-based, point cloud-based, and fusion-based. Specifically, the fusion-based branch employs a dual-stage pipeline, leveraging the latent correlation between the two modalities for information interaction and conducting channel-level fusion. Moreover, implicit alignment in the fusion branch, coupled with information supplementation from the two single-modal branches, better addresses the information loss caused by limited overlap in their field-of-view (FoV). In extensive experiments on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI), North Campus Long-Term (NCLT), USVInland datasets, and the real-world campus test, the proposed method stands out in precision-recall curves with an average maximum F1 score of 0.949, demonstrating superior robustness and generalization compared with other state-of-the-art methods.