Modeling Kansei Index for Images and Impression Estimation Using Fine Tuning

Abstract

In this study, we propose an effective method for estimating image impressions that is based on a Kansei (affective) index and then show how a deep learning model can be used to evaluate the suitability of images based on that Kansei index. To accomplish this, we exhaustively collected words and phrases that express image impressions using the evaluation grid method and then conducted an evaluation experiment for those collected words using Yahoo! Cloud Sourcing. Next, factor analysis was performed on the collected evaluation data, and the obtained factor scores were used as the Kansei index. Finally, we used ResNet18 trained with ImageNet to fine-tune each image's factor scores for use as supervised data and confirmed that our deep learning model could infer an effective Kansei index that correlates strongly with the target images.