Emotion Discrimination of Amusement Based on Three-Dimensional Data of Body Movements

Abstract

An important problem associated with the progression of aging in society is the improvement of quality of life (QOL). If we can maintain or improve the QOL for individuals, including the elderly, this will contribute not only to the revitalization and stability of society [1], but also to the reduction of social costs such as nursing care and medical expenses [2]. Therefore, a living support system, such as an acceptable living environment and health monitoring system, is required. There are currently no imaging systems capable of automatically and quantitatively evaluating the factors that improve individual QOL and the intensity thereof in real time. Human QOL and "laughs/smiles" are closely related. Individuals who laugh every day exhibit better health and improved psychological state. In addition, calmness is recognized in animal-assisted therapy and is known to contribute to the improvement of QOL [3]. Based on these fi ndings, it is considered that maintaining or improving QOL is possible in an environment where amusement and calm are encouraged. Using visual images, it is difficult to distinguish between natural facial expressions and those shown intentionally; however, there is no example for defining and observing calmness [4]. The report on emotions by Miyasaka et al. focused on changes in facial skin temperature and accompanying blood fl ow [5] with emphasis on nose temperature, while Kumamoto et al. reported on a method for evaluating stress [6]. There is no system that considers an example of integrated visible and infrared images to evaluate attitude changes. In our previous studies, we showed that movement features of the lip change as the subject's feelings (state of no stress, no vision, etc.,) and physical conditions change, while acquiring motion data on the movement of the lips. We clarifi ed that the presence or absence of psychological changes can be discriminated by the movement features of the lip when amusement is evoked [7]. Vertical motion was observed in the head and shoulders when a strong emotion such as "amusement" was expressed. Nonverbal communication, including body movements, contributes to the transmission of approximately 65% of information [8]. That is, combining facial expression, movement features of the lips, and movement features of the body is useful for the recognition and quantification of emotions. In this paper, we develop elementary technologies of multiimage processing systems that can recognize multiple emotions by focusing on the basic study of body movement features to recognize emotion. This study aims to verify whether emotions of amusement can be detected with body movement features by using Microsoft Kinect. Although there are individual differences, this study aims to demonstrate the possibility of using body movement features as indicators of the strength of emotional expressions. Specifi cally, we use the XBOX ONE KINECT SENSOR (Microsoft Corp, Kinect for short), which focuses on nonverbal information such as body movements, as well as human movement in a noncontact fashion while watching an emotion-eliciting video [9]. We acquired threedimensional data on head and shoulder movements while subjects were viewing an emotion-eliciting video, and we conducted studies on the quantifi cation of amusement and calmness. In this paper, we examine the correlation between emotions of "amusement" and body movements through multiple methods, using data acquired from multiple subjects. Emotion Discrimination of Amusement Based on Three-Dimensional Data of Body Movements