Aesthetic Image Synthesis Using Multiple-Aesthetic-Aware GAN

Abstract

The use of synthesized images in fields that prioritize visual appearance, particularly the field of art, places a high value on aesthetics, with the system's output images required to adhere to several aesthetic rules. There are two approaches to aesthetic image synthesis with architecture based on GAN. The first strategy is to modify the loss function on the GAN so that, in addition to the loss from the GAN architecture, aesthetic loss and content/semantic loss are also calculated and the total loss is the sum of the three loss calculations. However, the outcomes still fall short of the expected natural image appearance. The second strategy involves modifying the GAN architecture by adding a new layer to the GAN generator and discriminator network, while the loss function calculation remains unchanged. The results of this second approach to image synthesis have not been optimized in terms of producing meaningful images across multiple semantic classes. This study proposes a method for increasing the aesthetic value of the synthesized image by modifying the two approaches and employing the multiple-aesthetic-aware GAN method. The proposed method takes conditional semantic information and conditional aesthetic information into account not only in the GAN architecture, but also in the loss function value calculation. The proposed method is the result of ongoing research and will be evaluated using the Inception Score (IS), the Frechet Inception Distance (FID), and an aesthetic value metric.