Physically-based Rendering of Animated Point Clouds for eXtended Reality

Abstract Point cloud 3D models are becoming more and more popular thanks to the spreading of scanning systems employed in many fields. When used for rendering purposes, point clouds are usually displayed with their original color acquired at scan time, without considering the lighting condition of the scene where the model is placed. This leads to a lack of realism in many contexts, especially in the case of animated point clouds employed in eXtended Reality applications where it would be desirable to have the model reacting to incoming light and integrating with the surrounding environment. This paper proposes the application of Physically Based Rendering (PBR), a rendering technique widely used in Real-Time Computer Graphics applications, to animated point cloud models for reproducing specular reflections, and achieving a photo-realistic and physically accurate look under any lighting condition. Firstly, we consider the extension of commonly used animated point cloud formats, to include normal vectors, and PBR parameters, as well as the encoding of the animated environment maps required by the technique. Then, an animated point cloud model is rendered with a shader implementing the proposed PBR method. Finally, the PBR pipeline is compared to traditional renderings of the same point cloud obtained with commonly used shaders, under different lighting conditions and environments. It will be shown how the point cloud better integrates visually with its surroundings.