Chunk-wise face model based gaze correction in conversational videos with single camera

Eye contact is one of critical aspects of video conference. In current video conference systems, an important problem is the shortage of eye contact. This is due to the direction disparity between the normal of focus plane of camera and the interlocutor's gaze. Currently, the state-of-the-method performs well on gaze correction with only single webcam. However, this method that applied to the sequences with face obscured has shown unsatisfying results. In this paper, we propose a novel model-based gaze correction scheme to independently build the eye models, the nose model and the mouth model independently so that the occlusion problem can be solved effectively. To be more specific, by rotating the aforementioned four models at a pre-detected displacement, the degree of gaze correction can be obtained. The facial appearance that is occluded will not be rotated, which will prevent the occluding object from being distorted. Experiment results show that, under different parts of occluding objects, our method can achieve better performance on gaze correction than the current method in terms of perceived quality in two scenes usually appearing in conversational videos.