Exemplar-based video inpainting approach using temporal relationship of consecutive frames

Abstract

Digital inpainting is a technique used to remove some specified objects or to repair damaged area in an image or a video file. It has a wide used range in many applications such as in film and entertainment production, digital archives repairing, and satellite photography making. The main challenge in video inpainting is the patched video sequence ought to remain as much visual quality as the original one. To meet this requirement, video inpainting must have a robust object tracking algorithm with considering factors of the continuity of temporal relationship between frames, especially for camera-moving cases. In this paper, we propose a video inpainting algorithm based on the exemplar-based method [2]. In the proposed method, we first use Harris corner detection to extract feature points of each frame and match them between consecutive frames. The frames with same feature points are aligned by using affine transformation, and the median filter method is then used to establish a dynamic panoramic background. Next, we propose a robust object tracking method by using three-steps searching algorithm wherein moving objects can be find in different frames. Then, the foreground objects of each frame are removed by using background subtraction. Finally, we use an exemplar-based video inpainting method to patch the video. Experimental results showed that the proposed method can not only accurately track the moving objects but also can repair the video without losing linear structure of image (frame) and will not produce blur. In sum, the proposed method is an efficient and effective video patching method that repairs the video of acceptable quality.