Multimodal Approach for Classifying Stress using Facial Emotion and Physiological Sensors

Abstract

The focus is on understanding emotional stress by itself may enhance artificial intelligence agents involved in emotion detection and human computer interaction. These emotional responses are reflected into emotions and facial expressions. This research work proposes a study of Stress Classification using both facial expression and Physiological Sensors. For getting facial data, transfer learning is used with fine-tuning to extract features from facial images. In transfer learning different Deep Learning architectures like VGG-19[13], ResNet are used. From the sensors data on four features are collected that is age, gender, body temperature and heartbeat and accordingly choose the architecture for doing stress classification. Finally, both these models are integrated for getting better results.