Facial expression recognition using two-tier classification and its application to smart home automation system

Abstract

With the convergence of smart technologies and advancement in electronic equipment the concept of smart home system swiftly escalates. The idea is to automate the home appliances according to the user requirements without human intervention. After a long tiring day and heavy workloads user will not be in a state of taking out its mobile phone and pressing the buttons for controlling home appliances. Several methods have been proposed in the design of such systems using sensors, biometrics and face detection. This paper proposes a method for detecting human emotions by taking into account the complete facial analysis, suggesting that the emotions can accurately be determined by analyzing eyes, nose and lips separately hence covering a wide range of emotions. The classification is carried out by acquiring the image of user followed by the face detection and segmenting the region of interests (ROI) i.e. eyes, nose and lips for further analysis of emotions. Principle Component Analysis (PCA) along with feature extraction techniques and Support Vector Machines (SVMs) are used for classification of emotion for the said automation system. Policies have been implemented in Java to simulate the home automation environment for testing and validation. At the instant this system has been tested on a single user with 4 basic emotions i.e. sad, anger, happiness and neutral, but this study can be a basis to develop an automated system with variety of emotions for multiple users.