A Content-Aware Full-Reference Image Quality Assessment Method Using a Gram Matrix and Signal-to-Noise

Abstract

With the emergence of transformer-based feature extractors, the effect of image quality assessment (IQA) has improved, but its interpretability is limited. In addition, images repaired by generative adversarial networks (GANs) produce realistic textures and spatial misalignments with high-quality images. In this paper, we develop a content-aware full-reference IQA method without changing the original convolutional neural network feature extractor. First, image signal-to-noise (SNR) mapping is performed experimentally to verify its superior content-aware ability, and based on the SNR mapping of the reference image, we fuse multiscale distortion and normal image features according to a fusion strategy that enhances the informative area. Second, judging the quality of GAN-generated images from the perspective of focusing on content may ignore the alignment between pixels; therefore, we add a Gram-matrix-based texture enhancement module to boost the texture information between distorted and normal difference features. Finally, experiments on numerous public datasets prove the superior performance of the proposed method in predicting image quality.