FreqSpace-NeRF: A fourier-enhanced Neural Radiance Fields method via dual-domain contrastive learning for novel view synthesis

Abstract

Inspired by Neural Radiance Field’s (NeRF) groundbreaking success in novel view synthesis, current methods mostly employ variants of various deep neural network architectures, and use the combination of multi-scale feature maps with the target viewpoint to synthesize novel views. However, these methods only consider spatial domain features, inevitably leading to the loss of some details and edge information. To address these issues, this paper innovatively proposes the FreqSpace-NeRF (FS-NeRF), aiming to significantly enhance the rendering fidelity and generalization performance of NeRF in complex scenes by integrating the unique advantages of spectral domain and spatial domain deep neural networks, and combining contrastive learning driven data augmentation techniques. Specifically, the core contribution of this method lies in designing a dual-stream network architecture: on one hand, capturing global frequency features through Fourier transformation; on the other hand, finely refining local details using well-established spatial domain convolutional neural networks. Moreover, to ensure the model can more acutely distinguish subtle differences between different views, we propose two loss functions: Frequency-Space Contrastive Entropy Loss (FSCE Loss) and Adaptive Spectral Contrastive Loss (ASC Loss). This innovation aims to more effectively guide the data flow and focuses on minimizing the frequency discrepancies between different views. By comprehensively utilizing the fusion of spectral and spatial domain features along with contrastive learning, FS-NeRF achieves significant performance improvements in scene reconstruction tasks. Extensive qualitative and quantitative evaluations confirm that our method surpasses current state-of-the-art (SOTA) models in this field.