STNeRF: symmetric triplane neural radiance fields for novel view synthesis from single-view vehicle images

This paper presents STNeRF, a method for synthesizing novel views of vehicles from single-view 2D images without the need for 3D ground truth data, such as point clouds, depth maps, CAD models, etc., as prior knowledge. A significant challenge in this task arises from the characteristics of CNNs and the utilization of local features can lead to a flattened representation of the synthesized image when training and validation with images from a single viewpoint. Many current methodologies tend to overlook local features and rely on global features throughout the entire reconstruction process, potentially resulting in the loss of fine-grained details in the synthesized image. To tackle this issue, we introduce Symmetric Triplane Neural Radiance Fields (STNeRF). STNeRF employs a triplane feature extractor with spatially aware convolution to extend 2D image features into 3D. This decouples the appearance component, which includes local features, and the shape component, which consists of global features, and utilizes them to construct a neural radiance field. These neural priors are then employed for rendering novel views. Furthermore, STNeRF leverages the symmetric properties of vehicles to liberate the appearance component from reliance on the original viewpoint and to align it with the symmetry of the target space, thereby enhancing the neural radiance field network’s ability to represent the invisible regions. The qualitative and quantitative evaluations demonstrate that STNeRF outperforms existing solutions in terms of both geometry and appearance reconstruction. More supplementary materials and the implementation code are available for access at the following link: https://github.com/ll594282475/STNeRF.