A tightly-coupled dense monocular Visual-Inertial Odometry system with lightweight depth estimation network

Abstract

In various fields such as unmanned aerial vehicles (UAVs) and autonomous driving, monocular dense Simultaneous Localization and Mapping (SLAM) and Visual Odometry (VO) allow devices of above mentioned fields to estimate their position and orientation in real-time while constructing dense maps, relying solely on a single camera sensor. However, existing solutions for dense SLAM/VO systems often come with high computational costs and lead to issues, such as scale drift and reduced localization accuracy, making them less practical than their sparse counterparts. We present MVS-VIO, a novel dense monocular visual inertial odometry system composed of two main components: real-time pose estimation and global Truncated Signed Distance Function (TSDF) reconstruction. The first component is LW-MVSNET, a lightweight multi-view depth estimation network that utilizes only three views and 68 depth hypotheses. The adaptive view aggregation (AVA) and adaptive depth hypotheses (ADH) modules can effectively reject inaccurate depth estimation results, preventing significant error accumulation during runtime by adopting an uncertainty mask. The second is a tightly-coupled optimization method leveraging a deep photometric error. To address the problem of underutilization of information due to a delayed generation of depth estimation, we incorporate a delayed marginalization strategy to optimize all the variables. LW-MVSNET is trained on the Replica dataset and performs good generalization on the TUM-RGBD and the EuRoC datasets, and the ablation study further validates the effectiveness of our modules. Notably, in all real-world sequences of the EuRoC dataset, our proposed MVS-VIO system outperforms comparable dense monocular systems. It operates stably in all eleven sequences at a rate of 10.08 frames per second (FPS), and achieves an average absolute trajectory error (ATE) of 0.066 meters, which represents state-of-the-art performance. This demonstrates that our method can reconstruct dense maps in real-time while maintaining a level of accuracy comparable to that of sparse systems.