Towards Robust Evaluation of Aesthetic and Photographic Quality Metrics: Insights from a Comprehensive Dataset

Abstract

In recent years, the development of metrics to evaluate image aesthetics and photographic quality has proliferated. However, validating these metrics presents challenges due to the inherently subjective nature of aesthetics and photographic quality, which can be influenced by cultural contexts and individual preferences that evolve over time. This article presents a novel validation methodology utilizing a dataset assessed by individuals from two distinct nationalities: the United States and Spain. Evaluation criteria include photographic quality and aesthetic value, with the dataset comprising images previously rated on the DPChallenge photographic portal. We analyze the correlation between these values and provide the dataset for future research endeavors. Our investigation encompasses several metrics, including BRISQUE for assessing photographic quality, NIMA aesthetic and NIMA technical for evaluating both aesthetic and technical aspects, Diffusion Aesthetics (employed in Stable Diffusion), and PhotoILike for gauging the commercial appeal of real estate images. Our findings reveal a significant correlation between the Diffusion Aesthetics metric and aesthetic measures, as well as with the NIMA aesthetics metric, suggesting them as good potential candidates to capture aesthetic value.