BE-SGGAN: Content-aware bit-depth enhancement by semantic guided GAN

Abstract

Bit-depth enhancement (BDE) is a potential and important way to improve the visual quality of low bit-depth (LBD) images when displayed on high bit-depth (HBD) monitors. With the rapid development of display technology, the demand for high-performance BDE algorithms is increasing. Although recent deep learning methods can reconstruct HBD images of better perceptual quality, they generally fail to recover realistic textures faithful to semantic classes and suffer from false contour artifacts in flat area, since they treat pixels in an indiscriminate way regardless of the semantic information. In this paper, we propose a novel content-aware semantic guided method to reconstruct photo-realistic HBD images by using Generative Adversarial Network (GAN). In particular, the framework of our model consists of a semantic guided generator as well as a semantic conditional discriminator. The semantic guided residual blocks (SGRBs) in our generator can perform pixel-level feature modulation conditioned on semantic segmentation map of the input LBD image to restore more realistic HBD image. The discriminator cascades the image and semantic segmentation map as input, and has an auxiliary semantic classification branch that determines whether the generated textures are consistent with the semantic categorical priors for superior discrimination performance. Besides, we take advantage of the semantic structural prior and introduce a novel gradient loss differentiating the flat areas against texture areas to further suppress the false contours in flat areas. Experiments show that our method has the ability of reconstructing natural and realistic HBD images.