Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666351
Nikhil Kaushik, Reynold Bailey, Alexander Ororbia, Cecilia Ovesdotter Alm
Confusion is a complex affective experience that involves both emotional and cognitive components, being less conspicuous than core emotions such as anger or sadness. We discuss an online data collection study designed to elicit confusion in spontaneous conversations across two dialogue tasks. Results from an analysis of the multimodal data (transcribed spoken language and facial expressions) suggest that the tasks induced naturalistic confusion, towards automated confusion recognition.
{"title":"Eliciting Confusion in Online Conversational Tasks","authors":"Nikhil Kaushik, Reynold Bailey, Alexander Ororbia, Cecilia Ovesdotter Alm","doi":"10.1109/aciiw52867.2021.9666351","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666351","url":null,"abstract":"Confusion is a complex affective experience that involves both emotional and cognitive components, being less conspicuous than core emotions such as anger or sadness. We discuss an online data collection study designed to elicit confusion in spontaneous conversations across two dialogue tasks. Results from an analysis of the multimodal data (transcribed spoken language and facial expressions) suggest that the tasks induced naturalistic confusion, towards automated confusion recognition.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132552490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666396
Juan Vazquez-Rodriguez
Having devices capable of understanding human emotions will significantly improve the way people interact with them. Moreover, if those devices are capable of influencing the emotions of users in a positive way, this will improve their quality of life, especially for frail or dependent users. A first step towards this goal is improving the performance of emotion recognition systems. Specifically, using a multimodal approach is appealing, as the availability of different signals is growing. We believe that it is important to incorporate new architectures and techniques like the Transformer and BERT, and to investigate how to use them in a multimodal setting. Also, it is essential to develop self-supervised learning techniques to take advantage of the considerable quantity of unlabeled data available nowadays. In this extended abstract, we present our research in those directions.
{"title":"Using Multimodal Transformers in Affective Computing","authors":"Juan Vazquez-Rodriguez","doi":"10.1109/aciiw52867.2021.9666396","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666396","url":null,"abstract":"Having devices capable of understanding human emotions will significantly improve the way people interact with them. Moreover, if those devices are capable of influencing the emotions of users in a positive way, this will improve their quality of life, especially for frail or dependent users. A first step towards this goal is improving the performance of emotion recognition systems. Specifically, using a multimodal approach is appealing, as the availability of different signals is growing. We believe that it is important to incorporate new architectures and techniques like the Transformer and BERT, and to investigate how to use them in a multimodal setting. Also, it is essential to develop self-supervised learning techniques to take advantage of the considerable quantity of unlabeled data available nowadays. In this extended abstract, we present our research in those directions.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116728641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/ACIIW52867.2021.9666244
Jannes Bützer, Ronald Böck
This paper aims on the investigation and recognition of upper-body movements during a naturalistic Human-Machine Interaction, in which humans interact with a technical system while sitting in front of it. Therefore, we focus on the Last Minute Corpus, that provides such a naturalistic scenario in combination with multimodal recordings. For feature extraction an approach called Probabilistic Breadth Features was used, allowing a condensed investigation of movement patterns. Finally, the classification was based on Extreme Learning Machines, comparing features obtained in three different conditions: the Kinect's spine point, head point, and a combination of both. In context of this naturalistic interaction setting, a mean accuracy of 86.1% was achieved.
{"title":"Comparison of Head and Body Movement Patterns in Naturalistic Human-Machine Interaction","authors":"Jannes Bützer, Ronald Böck","doi":"10.1109/ACIIW52867.2021.9666244","DOIUrl":"https://doi.org/10.1109/ACIIW52867.2021.9666244","url":null,"abstract":"This paper aims on the investigation and recognition of upper-body movements during a naturalistic Human-Machine Interaction, in which humans interact with a technical system while sitting in front of it. Therefore, we focus on the Last Minute Corpus, that provides such a naturalistic scenario in combination with multimodal recordings. For feature extraction an approach called Probabilistic Breadth Features was used, allowing a condensed investigation of movement patterns. Finally, the classification was based on Extreme Learning Machines, comparing features obtained in three different conditions: the Kinect's spine point, head point, and a combination of both. In context of this naturalistic interaction setting, a mean accuracy of 86.1% was achieved.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117135506","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666319
M. Baev, A. Gusev, A. Kremlev
We developed a research software which allows users to accurately detect FACS AUs and basic emotion expressions. This software was developed as a comprehensive FACS based measurement tool. Due to their inherent limitations we don't use any kind of neural network facial expression classification. We created five author's computer vision procedures and a set of logical rules to detect 18 AUs and seven basic emotion expressions. The software could evaluate both macro- and microexpressions. As evaluation results we provided three examples of analyzing data taken from the SAMM and CASME II databases using F2F Emotion Studio software.
{"title":"Unbiased Mimic Activity Evaluation: F2F Emotion Studio Software","authors":"M. Baev, A. Gusev, A. Kremlev","doi":"10.1109/aciiw52867.2021.9666319","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666319","url":null,"abstract":"We developed a research software which allows users to accurately detect FACS AUs and basic emotion expressions. This software was developed as a comprehensive FACS based measurement tool. Due to their inherent limitations we don't use any kind of neural network facial expression classification. We created five author's computer vision procedures and a set of logical rules to detect 18 AUs and seven basic emotion expressions. The software could evaluate both macro- and microexpressions. As evaluation results we provided three examples of analyzing data taken from the SAMM and CASME II databases using F2F Emotion Studio software.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114705944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666357
Raiyan Abdul Baten
As the prevalence of automation increases, creativity will play an ever-larger role in the tasks humans accomplish. In this dissertation, we first explore empirically how a social net-work's connectivity patterns and creative outcomes are affected by factors such as creative performance, popularity, and identity attributes of people. Accordingly, we seek to devise intelligent intervention approaches that can harness the empirical insights to optimize network-wide creative outcomes. We envision our work to inform not only managerial and algorithmic decision-making, but also public policy as it relates to helping humans become more creatively productive in a social network.
{"title":"Fantastic Ideas and Where to Find Them: Elevating Creativity in Self-organizing Social Networks","authors":"Raiyan Abdul Baten","doi":"10.1109/aciiw52867.2021.9666357","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666357","url":null,"abstract":"As the prevalence of automation increases, creativity will play an ever-larger role in the tasks humans accomplish. In this dissertation, we first explore empirically how a social net-work's connectivity patterns and creative outcomes are affected by factors such as creative performance, popularity, and identity attributes of people. Accordingly, we seek to devise intelligent intervention approaches that can harness the empirical insights to optimize network-wide creative outcomes. We envision our work to inform not only managerial and algorithmic decision-making, but also public policy as it relates to helping humans become more creatively productive in a social network.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123462762","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666424
Mirella Hladký, T. Schneeberger, Patrick Gebhard
Computational emotion recognition focuses on observable expressions. In the case of highly unpleasant emotions that are rarely displayed openly and mostly unconsciously regulated - such as shame - this approach can be difficult. In previous studies, we found participants to smile and laugh while experiencing shame. Most current approaches interpret smiles and laughter as signals of enjoyment. They neglect the internal emotional experience and the complexity of social signals. We present a planned mixed-methods study that will investigate underlying functions of smiles and laughter in shameful situations and how those reflect in the morphology of expression. Participants' smiles and laughter during shame-eliciting situations will be analyzed using behavioral observations. Semi-structured interviews will investigate their functions. The gained knowledge can improve computational emotion recognition and avoid misinterpretations of smiles and laughter. In the scope of the open science initiative, we describe the planned study in detail with its research questions, hypotheses, design, methods, and analyses.
{"title":"Understanding Shame Signals: Functions of Smile and Laughter in the Context of Shame","authors":"Mirella Hladký, T. Schneeberger, Patrick Gebhard","doi":"10.1109/aciiw52867.2021.9666424","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666424","url":null,"abstract":"Computational emotion recognition focuses on observable expressions. In the case of highly unpleasant emotions that are rarely displayed openly and mostly unconsciously regulated - such as shame - this approach can be difficult. In previous studies, we found participants to smile and laugh while experiencing shame. Most current approaches interpret smiles and laughter as signals of enjoyment. They neglect the internal emotional experience and the complexity of social signals. We present a planned mixed-methods study that will investigate underlying functions of smiles and laughter in shameful situations and how those reflect in the morphology of expression. Participants' smiles and laughter during shame-eliciting situations will be analyzed using behavioral observations. Semi-structured interviews will investigate their functions. The gained knowledge can improve computational emotion recognition and avoid misinterpretations of smiles and laughter. In the scope of the open science initiative, we describe the planned study in detail with its research questions, hypotheses, design, methods, and analyses.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131319608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666318
G. Dray, Pierre-Antoine Jean, Yann Maheu, J. Montmain, Nicolas Sutton-Charani
This paper describes some machine learning methods that we have implemented to participate in the AffectMove challenge which aims to develop technologies for classification of body movements in the areas of physical rehabilitation of chronic pain, mathematical problem solving and interactive dance contexts. The methods and results obtained are presented as well as some futureworks.
{"title":"The AffectMove Challenge: some machine learning approaches","authors":"G. Dray, Pierre-Antoine Jean, Yann Maheu, J. Montmain, Nicolas Sutton-Charani","doi":"10.1109/aciiw52867.2021.9666318","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666318","url":null,"abstract":"This paper describes some machine learning methods that we have implemented to participate in the AffectMove challenge which aims to develop technologies for classification of body movements in the areas of physical rehabilitation of chronic pain, mathematical problem solving and interactive dance contexts. The methods and results obtained are presented as well as some futureworks.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115822944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/ACIIW52867.2021.9666238
Sujeong Kim, Abhinav Garlapati, Jonah Lubin, Amir Tamrakar, Ajay Divakaran
We present a series of two studies conducted to understand user's affective states during voice-based human-machine interactions. Emphasis is placed on the cases of communication errors or failures. In particular, we are interested in understanding “confusion” in relation with other affective states. The studies consist of two types of tasks: (1) related to communication with a voice-based virtual agent: speaking to the machine and understanding what the machine says, (2) non-communication related, problem-solving tasks where the participants solve puzzles and riddles but are asked to verbally explain the answers to the machine. We collected audio-visual data and self-reports of affective states of the participants. We report results of two studies and analysis of the collected data. The first study was analyzed based on the annotator's observation, and the second study was analyzed based on the self-report.
{"title":"Towards Understanding Confusion and Affective States Under Communication Failures in Voice-Based Human-Machine Interaction","authors":"Sujeong Kim, Abhinav Garlapati, Jonah Lubin, Amir Tamrakar, Ajay Divakaran","doi":"10.1109/ACIIW52867.2021.9666238","DOIUrl":"https://doi.org/10.1109/ACIIW52867.2021.9666238","url":null,"abstract":"We present a series of two studies conducted to understand user's affective states during voice-based human-machine interactions. Emphasis is placed on the cases of communication errors or failures. In particular, we are interested in understanding “confusion” in relation with other affective states. The studies consist of two types of tasks: (1) related to communication with a voice-based virtual agent: speaking to the machine and understanding what the machine says, (2) non-communication related, problem-solving tasks where the participants solve puzzles and riddles but are asked to verbally explain the answers to the machine. We collected audio-visual data and self-reports of affective states of the participants. We report results of two studies and analysis of the collected data. The first study was analyzed based on the annotator's observation, and the second study was analyzed based on the self-report.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130308257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666337
Janet Wessler
This experimental research investigated if there are economic and social sanctions (i.e., backlash) for counter-stereotypically behaving, angry females in computer-mediated negotiations. Participants (N = 82) received angry or joyful chat messages from their ostensible male or female opposite (i.e., a computer program). Results confirm the well-known anger effect: Participants demanded lower points for themselves when negotiating with an angry vs. joyful opposite. Moreover, participants liked the angry opposite less, perceived them as less competent and more competitive. However, the opposite's gender did not moderate these findings, although exploratory evidence for a backlash effect emerged: Angry females had descriptively lower negotiation outcomes, were liked less and perceived as significantly more competitive than angry males. These results suggest that when studying negotiations in human-agent interactions, both emotions and gender should be considered as important factors driving negotiation results and social perceptions of the agent.
{"title":"Economic and Social Consequences of Anger and Gender in Computer-Mediated Negotiations: Is there a Backlash Against Angry Females?","authors":"Janet Wessler","doi":"10.1109/aciiw52867.2021.9666337","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666337","url":null,"abstract":"This experimental research investigated if there are economic and social sanctions (i.e., backlash) for counter-stereotypically behaving, angry females in computer-mediated negotiations. Participants (N = 82) received angry or joyful chat messages from their ostensible male or female opposite (i.e., a computer program). Results confirm the well-known anger effect: Participants demanded lower points for themselves when negotiating with an angry vs. joyful opposite. Moreover, participants liked the angry opposite less, perceived them as less competent and more competitive. However, the opposite's gender did not moderate these findings, although exploratory evidence for a backlash effect emerged: Angry females had descriptively lower negotiation outcomes, were liked less and perceived as significantly more competitive than angry males. These results suggest that when studying negotiations in human-agent interactions, both emotions and gender should be considered as important factors driving negotiation results and social perceptions of the agent.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134389476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-28DOI: 10.1109/aciiw52867.2021.9666277
C. Athanasiadis, E. Hortal, Stelios Asteriadis
Emotion recognition through audio is a rather challenging task that entails proper feature extraction and classification. Meanwhile, state-of-the-art classification strategies are usually based on deep learning architectures. Training complex deep learning networks normally requires very large audiovisual corpora with available emotion annotations. However, such availability is not always guaranteed since harvesting and annotating such datasets is a time-consuming task. In this work, temporal conditional Wasserstein Generative Adversarial Networks (tc-wGANs) are introduced to generate robust audio data by leveraging information from a face modality. Having as input temporal facial features extracted using a dynamic deep learning architecture (based on 3dCNN, LSTM and Transformer networks) and, additionally, conditional information related to annotations, our system manages to generate realistic spectrograms that represent audio clips corresponding to specific emotional context. As proof of their validity, apart from three quality metrics (Frechet Inception Distance, Inception Score and Structural Similarity index), we verified the generated samples applying an audio-based emotion recognition schema. When the generated samples are fused with the initial real ones, an improvement between 3.5 to 5.5% was achieved in audio emotion recognition performance for two state-of-the-art datasets.
{"title":"Temporal conditional Wasserstein GANs for audio-visual affect-related ties","authors":"C. Athanasiadis, E. Hortal, Stelios Asteriadis","doi":"10.1109/aciiw52867.2021.9666277","DOIUrl":"https://doi.org/10.1109/aciiw52867.2021.9666277","url":null,"abstract":"Emotion recognition through audio is a rather challenging task that entails proper feature extraction and classification. Meanwhile, state-of-the-art classification strategies are usually based on deep learning architectures. Training complex deep learning networks normally requires very large audiovisual corpora with available emotion annotations. However, such availability is not always guaranteed since harvesting and annotating such datasets is a time-consuming task. In this work, temporal conditional Wasserstein Generative Adversarial Networks (tc-wGANs) are introduced to generate robust audio data by leveraging information from a face modality. Having as input temporal facial features extracted using a dynamic deep learning architecture (based on 3dCNN, LSTM and Transformer networks) and, additionally, conditional information related to annotations, our system manages to generate realistic spectrograms that represent audio clips corresponding to specific emotional context. As proof of their validity, apart from three quality metrics (Frechet Inception Distance, Inception Score and Structural Similarity index), we verified the generated samples applying an audio-based emotion recognition schema. When the generated samples are fused with the initial real ones, an improvement between 3.5 to 5.5% was achieved in audio emotion recognition performance for two state-of-the-art datasets.","PeriodicalId":105376,"journal":{"name":"2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133687144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：