An integrated vision system on emotion understanding and identity confirmation

The technologies of computer vision provided the home robot with visual ability, is able to improve the friendly user experience in home robot application. The vision research tends to solve single problem for a particular area, such as emotion understanding, identity identification and object detection, etc. This paper proposed an integrated application of vision system for emotion and identity confirmation. The facial landmarks, RGB image, and skeleton information captured by Kinect are the input of integrated vision system. The facial landmarks are used for Action Unit (AU) detection since the emotion recognition is considered as a combination of action units. The RGB image and skeleton information are used for identity conformation. The main contributions are summarized as follows. 1) An integrated vision system is proposed for the user description. 2) Hierarchal-Architecture SVM is presented for analysis of facial action units. 3) The system uses facial image and skeleton information to enhance the ability of identity confirmation. Experiments are performed on online test with the average accuracy are obtained by 86.33% and 86.26%, respectively. The experimental results have demonstrated the effectiveness and efficiency of the proposed system in real time application.