Multi-Task Deep Neural Networks for Multimodal Personality Trait Prediction

Abstract

Artificial intelligence (AI) is being increasingly integrated into the hiring process. A prominent example is video interviews used by large organizations to quickly screen job candidates. The personality traits of job candidates, such as the Big Five characteristics, are predicted using computer vision and affective computing approaches. Past methods have used feature extraction, text analysis, and other multimodal methods to achieve a high prediction accuracy. We build upon past approaches by using a multi-task deep neural network (MTDNN) to predict personality traits and job interview scores of individuals. An MTDNN shares lower layers to learn features which apply across outputs, and contains task-specific layers to predict each individual trait, thereby providing an advantage over single-task approaches since personality traits are determined by features (e.g., emotion, gestures, and speech) shared across traits. Our model is trained using the CVPR 2017 First Impressions V2 competition dataset, containing 10,000 videos of individuals and their Big Five personality and interview scores. We also use scene, audio, and facial features from the state-of-the-art model from the competition. A 5-fold cross-validation approach is used to evaluate our results. We achieve a prediction accuracy for all traits on par with state-of-the-art models, while reducing training time and parameter tuning to a single network.