Monocular Depth Estimation on Adverse Weathers With Curriculum Domain Distribution Alignment

Abstract

Despite the remarkable success of monocular depth estimation, most works focus on ideal experiment conditions, such as favorable weather, where there is few environmental factors impacting the depth estimation system. In practical, when suffering from adverse weather conditions, such as fog and rain, the model trained on favorable weather degrades sharply as the domain shift, caused by the decreasing of visibility. To solve this problem, in this paper, we propose a Curriculum Domain Distribution Alignment (CDA) algorithm to learn the domain-invariant representation, progressively aligning data distributions across favorable weather and adverse weather in the feature space. Concretely, to construct a domain adaptation curriculum, we first separate the target domain into several subsets with increased domain discrepancy based on an optical model. Then, we bridge the distribution discrepancy between domains from easier to harder data by matching the source and target representation subspace. Furthermore, to control the distribution aligning pace, we introduce self-paced learning to learn a dynamic domain adaptation weight, promoting the generalization ability of monocular depth estimation networks against environmental factors. We conduct experiments with six monocular depth estimation frameworks on FoggyCityScapes, RainCityScapes, SnowCityscapes, and All-day Cityscapes, improving RMSE with 8.5 %, 30.5 %, 30.9 %, 20.9 %. The extraordinary performance demonstrates the effectiveness and generalizability of our method under adverse weather conditions.