Self-supervised Depth Completion with Adaptive Online Adaptation

Abstract

Although depth completion has achieved remarkable performance relying on deep learning in recent years, these models tend to suffer a performance degradation when exposed to new environments. Online adaptation, where the model is trained in a self-supervised manner during testing, seems a promising technique to alleviate the drop. However, continuous online adaptation may cause the model to over-adapt and miss the optimal parameters, resulting in oscillation or even degradation of the model performance, in addition to wasting computational resources. Therefore, this paper proposes an adaptive online adaptation framework to make model adaptively trigger online adaptation when encountering novel environments and stop adaptation when model has adapted to the current environment. In detail, we design a trigger to detect the familiarity of model to the current scenario based on image similarity and then launch online adaptation when the scenario is novel. Besides, we elaborate a stopper to monitor the error between prediction and depth input and convert online adaptation to inference when online adaptation does not bring improvement for model. Experimental results demonstrate that our method improves the accuracy of model prediction and increases average running speed of the model on each frame in online adaptation.