Realistic retargeting of facial video

Abstract

We propose a simple method for realistic retargeting of facial performance from one shot to another. Editors can combine different takes of a shot into one single, optimal take with minimal manual labour and highly realistic results. Using a static proxy mesh of the actor's head, we obtain approximate 3D information of recorded monocular facial video. This 3D information is used to create pose-invariant textures from recorded facial action and to re-render it into a target shot. This can be done for the full face or parts of it, allowing for flexible editing.