A Framework for Driver Emotion Recognition using Deep Learning and Grassmann Manifolds

Abstract

In this paper, we propose a novel, real-time, camera based framework for determining the drivers emotions through facial expression recognition. Studies have established that driver's emotions play an important role in driving behavior. Therefore, continuous monitoring of the driver's emotions and requisite warning to the driver will help in maintaining safety on the roads. In our framework, at regular intervals, we detect the driver's face in the current frame and recognize the driver's emotions. For expression recognition, we extract features from the face image using two standard pre-trained deep neural networks, AlexNet and VGG16, that we fine-tune on facial expression data. We extract the features from the fully connected layer from these two networks for each frame and concatenate the two feature vectors to form a single feature vector. The novelty of our framework lies in creating distinct subspaces of each expression, using these feature vectors and applying Grassmann graph embedding based discriminant analysis to recognize the expression. The subspaces accommodate the variations in multiple instances of an expression of the same person as well as across multiple people. Our experimental results on standard datasets show that our proposed framework outperforms state-of-the-art methods.