MT-NeRF: Neural implicit representation based on multi-resolution geometric feature planes

Abstract

Reconstructing an indoor-scale scene from scratch is a difficult task when the camera pose is unknown. If it is also required to achieve fast convergence without sacrificing quality and ensure low memory usage at the same time, this work will be even more challenging. In this paper, we propose MT-NeRF, a novel radiance field rendering method based on RGB-D inputs without pre-computed camera poses. MT-NeRF maps indoor scenes at real-world scales to multi-resolution geometric feature planes, which greatly reduces memory footprint and enhances detailed scene fitting. In addition, MT-NeRF significantly enhances the localization accuracy of the system by introducing a photometric distortion loss based on interframe surface pixels. For keyframe selection, MT-NeRF employs a global-to-local keyframe selection strategy, which markedly enhances the global consistency of scene reconstruction. Experiments are designed and conducted to validate the effectiveness of MT-NeRF in scenarios involving complex motion or noisy depth map inputs. The results demonstrate remarkable improvements in scene reconstruction quality and pose estimation accuracy, all while ensuring a low memory footprint. At the same time, our method achieves a speedup of approximately fivefold.