Yan Zhao, Ming Guo, Xuehan Sun, Xiangyong Chen, Feng Zhao
{"title":"卷积神经网络与加权核支持向量机相结合,利用惯性测量单元信号实现基于注意力的传感器融合,用于人体运动情感识别","authors":"Yan Zhao, Ming Guo, Xuehan Sun, Xiangyong Chen, Feng Zhao","doi":"10.1049/sil2.12201","DOIUrl":null,"url":null,"abstract":"<p>The remarkable development of human–computer interactions has created an urgent need for machines to be able to recognise human emotions. Human motions play a key role in emphasising and conveying emotions to meet the complexity of daily application scenarios, such as medical rehabilitation and social education. Therefore, this paper aims to explore hidden emotional states from human motions. Accordingly, we proposed a novel approach for emotion recognition using multiple inertial measurement unit (IMU) sensors worn on different body parts. First, the mapping relationship between emotion and human motion was established through fuzzy comprehensive evaluation, and data were collected for six emotional states: sleepy, bored, excited, tense, angry, and distressed. Second, the preprocessed data were used as input in a lightweight convolutional neural network to extract discriminative features. Third, an attention-based sensor fusion module was developed to obtain the importance scores of each IMU sensor for generating a fused feature representation. In the recognition phase, we constructed a weighted kernel support vector machine (SVM) model with an auxiliary fuzzy function to improve the weight calculation method of kernel functions in a multiple kernel SVM. Finally, the results obtained are compared with those of similar state-of-the-art studies, the proposed method showed a higher accuracy (99.02%) for the six emotional states mentioned above. These findings may promote the development of social robots with non-verbal emotional communication capabilities.</p>","PeriodicalId":56301,"journal":{"name":"IET Signal Processing","volume":null,"pages":null},"PeriodicalIF":1.1000,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/sil2.12201","citationCount":"2","resultStr":"{\"title\":\"Attention-based sensor fusion for emotion recognition from human motion by combining convolutional neural network and weighted kernel support vector machine and using inertial measurement unit signals\",\"authors\":\"Yan Zhao, Ming Guo, Xuehan Sun, Xiangyong Chen, Feng Zhao\",\"doi\":\"10.1049/sil2.12201\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>The remarkable development of human–computer interactions has created an urgent need for machines to be able to recognise human emotions. Human motions play a key role in emphasising and conveying emotions to meet the complexity of daily application scenarios, such as medical rehabilitation and social education. Therefore, this paper aims to explore hidden emotional states from human motions. Accordingly, we proposed a novel approach for emotion recognition using multiple inertial measurement unit (IMU) sensors worn on different body parts. First, the mapping relationship between emotion and human motion was established through fuzzy comprehensive evaluation, and data were collected for six emotional states: sleepy, bored, excited, tense, angry, and distressed. Second, the preprocessed data were used as input in a lightweight convolutional neural network to extract discriminative features. Third, an attention-based sensor fusion module was developed to obtain the importance scores of each IMU sensor for generating a fused feature representation. In the recognition phase, we constructed a weighted kernel support vector machine (SVM) model with an auxiliary fuzzy function to improve the weight calculation method of kernel functions in a multiple kernel SVM. Finally, the results obtained are compared with those of similar state-of-the-art studies, the proposed method showed a higher accuracy (99.02%) for the six emotional states mentioned above. These findings may promote the development of social robots with non-verbal emotional communication capabilities.</p>\",\"PeriodicalId\":56301,\"journal\":{\"name\":\"IET Signal Processing\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.1000,\"publicationDate\":\"2023-04-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1049/sil2.12201\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IET Signal Processing\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1049/sil2.12201\",\"RegionNum\":4,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IET Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1049/sil2.12201","RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Attention-based sensor fusion for emotion recognition from human motion by combining convolutional neural network and weighted kernel support vector machine and using inertial measurement unit signals
The remarkable development of human–computer interactions has created an urgent need for machines to be able to recognise human emotions. Human motions play a key role in emphasising and conveying emotions to meet the complexity of daily application scenarios, such as medical rehabilitation and social education. Therefore, this paper aims to explore hidden emotional states from human motions. Accordingly, we proposed a novel approach for emotion recognition using multiple inertial measurement unit (IMU) sensors worn on different body parts. First, the mapping relationship between emotion and human motion was established through fuzzy comprehensive evaluation, and data were collected for six emotional states: sleepy, bored, excited, tense, angry, and distressed. Second, the preprocessed data were used as input in a lightweight convolutional neural network to extract discriminative features. Third, an attention-based sensor fusion module was developed to obtain the importance scores of each IMU sensor for generating a fused feature representation. In the recognition phase, we constructed a weighted kernel support vector machine (SVM) model with an auxiliary fuzzy function to improve the weight calculation method of kernel functions in a multiple kernel SVM. Finally, the results obtained are compared with those of similar state-of-the-art studies, the proposed method showed a higher accuracy (99.02%) for the six emotional states mentioned above. These findings may promote the development of social robots with non-verbal emotional communication capabilities.
期刊介绍:
IET Signal Processing publishes research on a diverse range of signal processing and machine learning topics, covering a variety of applications, disciplines, modalities, and techniques in detection, estimation, inference, and classification problems. The research published includes advances in algorithm design for the analysis of single and high-multi-dimensional data, sparsity, linear and non-linear systems, recursive and non-recursive digital filters and multi-rate filter banks, as well a range of topics that span from sensor array processing, deep convolutional neural network based approaches to the application of chaos theory, and far more.
Topics covered by scope include, but are not limited to:
advances in single and multi-dimensional filter design and implementation
linear and nonlinear, fixed and adaptive digital filters and multirate filter banks
statistical signal processing techniques and analysis
classical, parametric and higher order spectral analysis
signal transformation and compression techniques, including time-frequency analysis
system modelling and adaptive identification techniques
machine learning based approaches to signal processing
Bayesian methods for signal processing, including Monte-Carlo Markov-chain and particle filtering techniques
theory and application of blind and semi-blind signal separation techniques
signal processing techniques for analysis, enhancement, coding, synthesis and recognition of speech signals
direction-finding and beamforming techniques for audio and electromagnetic signals
analysis techniques for biomedical signals
baseband signal processing techniques for transmission and reception of communication signals
signal processing techniques for data hiding and audio watermarking
sparse signal processing and compressive sensing
Special Issue Call for Papers:
Intelligent Deep Fuzzy Model for Signal Processing - https://digital-library.theiet.org/files/IET_SPR_CFP_IDFMSP.pdf