GEAST-RF: Geometry Enhanced 3D Arbitrary Style Transfer Via Neural Radiance Fields

Abstract

Style transfer techniques integrated with neural radiance fields enhance the stylization effect of the 3D scene. The objective of 3D style transfer is to render novel views of stylized 3D scenes while maintaining multi-view consistency. However, the current state of 3D style transfer confronts three principal challenges: precise geometric reconstruction, style bias issues, and the artifacts and floaters that frequently emerge during the stylization process. To address these issues, we propose GEAST-RF (Geometry Enhanced 3D Arbitrary Style Transfer Via Neural Radiance Fields), which employs explicit high-level feature grids to represent 3D scenes, achieving detailed geometry reconstruction through volume rendering and high-quality 3D arbitrary style transfer based on target style image information. Specifically, GEAST-RF introduces two pivotal innovations to enhance 3D stylization. The first is the geometry enhancements module, which aligns the geometric structures of stylized views from the same viewpoint to those in the content views, enabling high-precision geometry reconstruction. Thresholding and masking operations are introduced during alignment to alleviate artifacts such as floaters produced during rendering. The second is the adaptive stylization module, which utilizes adaptive computation during the stylization stage to make the model focus more on core style information, reducing reliance on edge style information. Our experiments demonstrate that GEAST-RF can achieve precise geometric structures while providing exceptional 3D stylization effects. A user survey further corroborates these experimental results, revealing that the majority of participants prefer our generated outputs compared to the most recent state-of-the-art methods.