Modified GAN for Natural Occlusion Detection and Inpainting of Raw Footage From Video Surveillance

Abstract

Low resolution and occlusion are mainly prominent in images taken from certain unconstrained environments such as raw footage from video surveillance. In this work, a deep generative adversarial network for joint face completion and face super-resolution is proposed. It will be really useful in the current COVID-19 scenario as people wearing masks are a common sight. Given an input of a low-resolution face image with occlusion, the generator aims to recover a high-resolution face image without occlusion. The discriminator uses a set of carefully designed losses to assure the high quality of the recovered high-resolution face images without occlusion. Experimental results on CelebA database show that the proposed approach outperforms the state-of-the-art methods in jointly performing face super-resolution and face completion, and shows good generalization ability in cross-database testing. MSSIM showed an accuracy of around 80% for test cases. The recorded values of generator adversarial loss, generator pixel loss, and discriminator loss are 0.93, 0.10, and 0.003, respectively.