Region Dependent Mesh Refinement for Volumetric Video Workflows

Abstract

This paper addresses high quality mesh optimization for volumetric video. Real persons are captured with multiple cameras and converted to 3D mesh sequences. These volumetric video assets can be used as dynamic 3D objects in arbitrary 3D rendering engines. In this way, 3D representations of real persons are achieved with a high level of detail and realism. Target use cases are augmented reality, virtual reality and mixed reality applications. However, the final rendering quality strongly depends on the hardware capabilities of the target rendering device. In this context, a novel region dependent mesh refinement approach is presented and evaluated with respect to existing workflows. The proposed approach is used in order to obtain a low overall polygon count while keeping details in semantically important regions such as human faces. It combines conventional 2D skin and face detection algorithms and transfers the results to the 3D domain. Further on, a dedicated camera region selection approach is presented which enhances the sharpness and quality of the resulting 3D texture mappings.