We address the task of automatically predicting group satisfaction in meetings using acoustic, lexical, and turn-taking features. Participant satisfaction is measured using post-meeting ratings from the AMI corpus. We focus on predicting three aspects of satisfaction: overall satisfaction, participant attention satisfaction, and information overload. All predictions are made at the aggregated group level. In general, we find that combining features across modalities improves prediction performance. However, feature ablation significantly improves performance. Our experiments also show how data-driven methods can be used to explore how different facets of group satisfaction are expressed through different modalities. For example, inclusion of prosodic features improves prediction of attention satisfaction but hinders prediction of overall satisfaction, but the opposite for lexical features. Moreover, feelings of sufficient attention were better reflected by acoustic features than by speaking time, while information overload was better reflected by specific lexical cues and turn-taking patterns. Overall, this study indicates that group affect can be revealed as much by how participants speak, as by what they say.
{"title":"Predicting group satisfaction in meeting discussions","authors":"Catherine Lai, Gabriel Murray","doi":"10.1145/3279810.3279840","DOIUrl":"https://doi.org/10.1145/3279810.3279840","url":null,"abstract":"We address the task of automatically predicting group satisfaction in meetings using acoustic, lexical, and turn-taking features. Participant satisfaction is measured using post-meeting ratings from the AMI corpus. We focus on predicting three aspects of satisfaction: overall satisfaction, participant attention satisfaction, and information overload. All predictions are made at the aggregated group level. In general, we find that combining features across modalities improves prediction performance. However, feature ablation significantly improves performance. Our experiments also show how data-driven methods can be used to explore how different facets of group satisfaction are expressed through different modalities. For example, inclusion of prosodic features improves prediction of attention satisfaction but hinders prediction of overall satisfaction, but the opposite for lexical features. Moreover, feelings of sufficient attention were better reflected by acoustic features than by speaking time, while information overload was better reflected by specific lexical cues and turn-taking patterns. Overall, this study indicates that group affect can be revealed as much by how participants speak, as by what they say.","PeriodicalId":326513,"journal":{"name":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117148592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Leah Friedman, Ruixue Liu, Erin Walker, E. Solovey
As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual's changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain's default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.
{"title":"Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes","authors":"Leah Friedman, Ruixue Liu, Erin Walker, E. Solovey","doi":"10.1145/3279810.3279854","DOIUrl":"https://doi.org/10.1145/3279810.3279854","url":null,"abstract":"As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual's changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain's default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.","PeriodicalId":326513,"journal":{"name":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","volume":"20 7","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132329899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Research in various fields including psychology, cognition, and medical science deal with eye tracking data to extract information about the intention and cognitive state of a subject. For the extraction of this information, the detection of eye movement types is an important task. Modern eye tracking data is noisy and most of the state-of-the-art algorithms are not developed for all types of eye movements since they are still under research. We propose a novel feature for eye movement detection, which is called histogram of oriented velocities. The construction of the feature is similar to the well known histogram of oriented gradients from computer vision. Since the detector is trained using machine learning, it can always be extended to new eye movement types. We evaluate our feature against the state-of-the-art on publicly available data. The evaluation includes different machine learning approaches such as support vector machines, regression trees, and k nearest neighbors. We evaluate our feature together with the machine learning approaches for different parameter sets. We provide a matlab script for the computation and evaluation as well as an integration in EyeTrace which can be downloaded at http://www.ti.uni-tuebingen.de/Eyetrace.1751.0.html.
{"title":"Histogram of oriented velocities for eye movement detection","authors":"Wolfgang Fuhl, Nora Castner, Enkelejda Kasneci","doi":"10.1145/3279810.3279843","DOIUrl":"https://doi.org/10.1145/3279810.3279843","url":null,"abstract":"Research in various fields including psychology, cognition, and medical science deal with eye tracking data to extract information about the intention and cognitive state of a subject. For the extraction of this information, the detection of eye movement types is an important task. Modern eye tracking data is noisy and most of the state-of-the-art algorithms are not developed for all types of eye movements since they are still under research. We propose a novel feature for eye movement detection, which is called histogram of oriented velocities. The construction of the feature is similar to the well known histogram of oriented gradients from computer vision. Since the detector is trained using machine learning, it can always be extended to new eye movement types. We evaluate our feature against the state-of-the-art on publicly available data. The evaluation includes different machine learning approaches such as support vector machines, regression trees, and k nearest neighbors. We evaluate our feature together with the machine learning approaches for different parameter sets. We provide a matlab script for the computation and evaluation as well as an integration in EyeTrace which can be downloaded at http://www.ti.uni-tuebingen.de/Eyetrace.1751.0.html.","PeriodicalId":326513,"journal":{"name":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129202275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Humans adapt their behaviour to the perceived cognitive load of their dialogue partner, for example, delaying non-essential information. We propose that spoken dialogue systems should do the same, particularly in high-stakes scenarios, such as emergency response. In this paper, we provide a summary of the prosodic, turn-taking and other linguistic symptoms of cognitive load analysed in the literature. We then apply these features to a single corpus in the restaurant-finding domain and propose new symptoms that are evidenced through interaction with the dialogue system, including utterance entropy, speech recognition confidence, as well as others based on dialogue acts.
{"title":"Symptoms of cognitive load in interactions with a dialogue system","authors":"José Lopes, K. Lohan, H. Hastie","doi":"10.1145/3279810.3279851","DOIUrl":"https://doi.org/10.1145/3279810.3279851","url":null,"abstract":"Humans adapt their behaviour to the perceived cognitive load of their dialogue partner, for example, delaying non-essential information. We propose that spoken dialogue systems should do the same, particularly in high-stakes scenarios, such as emergency response. In this paper, we provide a summary of the prosodic, turn-taking and other linguistic symptoms of cognitive load analysed in the literature. We then apply these features to a single corpus in the restaurant-finding domain and propose new symptoms that are evidenced through interaction with the dialogue system, including utterance entropy, speech recognition confidence, as well as others based on dialogue acts.","PeriodicalId":326513,"journal":{"name":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128428160","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","authors":"","doi":"10.1145/3279810","DOIUrl":"https://doi.org/10.1145/3279810","url":null,"abstract":"","PeriodicalId":326513,"journal":{"name":"Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126872519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：