MECA: Manipulation With Emotional Intensity-Aware Contrastive Learning and Attention-Based Discriminative Learning

Abstract

With recent developments in deep learning, facial expression manipulation (FEM) has become one of the fields receiving great attention. However, many studies focus on learning without considering class distinction in latent space. This paper introduces a representation learning scheme that leverages self-attention and mutual information to effectively account for semantic attributes, such as facial expressions, in the FEM task. Our framework, utilizing attention-based discriminative learning and emotional intensity-aware contrastive learning, is capable of forming a compact embedding space. This compact embedding space can lead to more discerning and richer facial expression synthesis in actual synthesis results. As a result, we have derived facial expression synthesis results that are superior to the previous methods. Also, in terms of the FED metric, which can quantify the degree of facial expression expression in FEM, the proposed method outperforms the other methods. To demonstrate this successful result, we use t-SNE and visualize the actual embedding results for each class. Furthermore, we prove that the latent space formed through the proposed method is also helpful in terms of facial expression recognition.