Zero-shot domain adaptation with enhanced consistency for semantic segmentation

Zero-shot domain adaptation is a specialized area within transfer learning focused on achieving domain adaptation without using any samples from the target domain. This is particularly important when target domain samples are difficult to obtain. The rapid development in generative models, particularly diffusion models, has introduced robust tools for zero-shot domain adaptation tasks. This paper proposes an innovative framework to address domain adaptive semantic segmentation under zero-shot conditions. We introduce a Dynamic Control Fusion Module, which autonomously learns the fusion scales and effectively integrates hidden states with image controls, enhancing generation in complex scenarios. Furthermore, we propose a Semantic and Image-Text Consistency Strategy, designed to impose consistency constraints on both the semantic content and the style of generated images, ensuring closer alignment with the target domain. We perform experiments on Cityscapes, ACDC, and GTAV datasets. The results show that our method improves the quality of generated target domain images and semantic segmentation performance, demonstrating its effectiveness in zero-shot domain adaptation tasks. Overall, our method shows consistent improvements over baseline approaches across the five sub-experiments. Overall, our method demonstrates consistent improvements over baseline approaches across most domain adaptation tasks. Specifically, in the tasks involving adaptation to Night and Snow, it achieves 2.6% and 2.3% higher mIoU compared to the baseline, respectively.