We introduce vlogs as a type of rich human interaction which is multimodal in nature and suitable for new large-scale behavioral data analysis. The automatic analysis of vlogs is useful not only to study social media, but also remote communication scenarios, and requires the integration of methods for multimodal processing and for social media understanding. Based on works from social psychology and computing, we first propose robust audio, visual, and multimodal cues to measure the nonverbal behavior of vloggers in their videos. Then, we investigate the relation between behavior and the attention videos receive in YouTube. Our study shows significant correlations between some nonverbal behavioral cues and the average number of views per video.
{"title":"Vlogcast yourself: nonverbal behavior and attention in social media","authors":"Joan-Isaac Biel, D. Gática-Pérez","doi":"10.1145/1891903.1891964","DOIUrl":"https://doi.org/10.1145/1891903.1891964","url":null,"abstract":"We introduce vlogs as a type of rich human interaction which is multimodal in nature and suitable for new large-scale behavioral data analysis. The automatic analysis of vlogs is useful not only to study social media, but also remote communication scenarios, and requires the integration of methods for multimodal processing and for social media understanding. Based on works from social psychology and computing, we first propose robust audio, visual, and multimodal cues to measure the nonverbal behavior of vloggers in their videos. Then, we investigate the relation between behavior and the attention videos receive in YouTube. Our study shows significant correlations between some nonverbal behavioral cues and the average number of views per video.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"176 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116506360","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hui Zhang, Damian Fricker, Thomas G. Smith, Chen Yu
Multimodal interaction in everyday life seems so effortless. However, a closer look reveals that such interaction is indeed complex and comprises multiple levels of coordination, from high-level linguistic exchanges to low-level couplings of momentary bodily movements both within an agent and across multiple interacting agents. A better understanding of how these multimodal behaviors are coordinated can provide insightful principles to guide the development of intelligent multimodal interfaces. In light of this, we propose and implement a research framework in which human participants interact with a virtual agent in a virtual environment. Our platform allows the virtual agent to keep track of the user's gaze and hand movements in real time, and adjust his own behaviors accordingly. An experiment is designed and conducted to investigate adaptive user behaviors in a human-agent joint attention task. Multimodal data streams are collected in the study including speech, eye gaze, hand and head movements from both the human user and the virtual agent, which are then analyzed to discover various behavioral patterns. Those patterns show that human participants are highly sensitive to momentary multimodal behaviors generated by the virtual agent and they rapidly adapt their behaviors accordingly. Our results suggest the importance of studying and understanding real-time adaptive behaviors in human-computer multimodal interactions.
{"title":"Real-time adaptive behaviors in multimodal human-avatar interactions","authors":"Hui Zhang, Damian Fricker, Thomas G. Smith, Chen Yu","doi":"10.1145/1891903.1891909","DOIUrl":"https://doi.org/10.1145/1891903.1891909","url":null,"abstract":"Multimodal interaction in everyday life seems so effortless. However, a closer look reveals that such interaction is indeed complex and comprises multiple levels of coordination, from high-level linguistic exchanges to low-level couplings of momentary bodily movements both within an agent and across multiple interacting agents. A better understanding of how these multimodal behaviors are coordinated can provide insightful principles to guide the development of intelligent multimodal interfaces. In light of this, we propose and implement a research framework in which human participants interact with a virtual agent in a virtual environment. Our platform allows the virtual agent to keep track of the user's gaze and hand movements in real time, and adjust his own behaviors accordingly. An experiment is designed and conducted to investigate adaptive user behaviors in a human-agent joint attention task. Multimodal data streams are collected in the study including speech, eye gaze, hand and head movements from both the human user and the virtual agent, which are then analyzed to discover various behavioral patterns. Those patterns show that human participants are highly sensitive to momentary multimodal behaviors generated by the virtual agent and they rapidly adapt their behaviors accordingly. Our results suggest the importance of studying and understanding real-time adaptive behaviors in human-computer multimodal interactions.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126889034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Kurihara, T. Mochizuki, Hiroki Oura, Mio Tsubakimoto, T. Nishimori, Jun Nakahara
In this paper we propose new quantitative metrics that express the characteristics of current general practices in slide-based presentation methodology. The proposed metrics are numerical expressions of: 'To what extent are the materials being presented in the prepared order?' and 'What is the degree of separation between the displays of the presenter and the audience?'. Through the use of these metrics, it becomes possible to quantitatively evaluate various extended methods designed to improve presentations. We illustrate examples of calculation and visualization for the proposed metrics.
{"title":"Linearity and synchrony: quantitative metrics for slide-based presentation methodology","authors":"K. Kurihara, T. Mochizuki, Hiroki Oura, Mio Tsubakimoto, T. Nishimori, Jun Nakahara","doi":"10.1145/1891903.1891947","DOIUrl":"https://doi.org/10.1145/1891903.1891947","url":null,"abstract":"In this paper we propose new quantitative metrics that express the characteristics of current general practices in slide-based presentation methodology. The proposed metrics are numerical expressions of: 'To what extent are the materials being presented in the prepared order?' and 'What is the degree of separation between the displays of the presenter and the audience?'. Through the use of these metrics, it becomes possible to quantitatively evaluate various extended methods designed to improve presentations. We illustrate examples of calculation and visualization for the proposed metrics.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128091469","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
I. D. Kok, Derya Ozkan, D. Heylen, Louis-Philippe Morency
Traditionally listener response prediction models are learned from pre-recorded dyadic interactions. Because of individual differences in behavior, these recordings do not capture the complete ground truth. Where the recorded listener did not respond to an opportunity provided by the speaker, another listener would have responded or vice versa. In this paper, we introduce the concept of parallel listener consensus where the listener responses from multiple parallel interactions are combined to better capture differences and similarities between individuals. We show how parallel listener consensus can be used for both learning and evaluating probabilistic prediction models of listener responses. To improve the learning performance, the parallel consensus helps identifying better negative samples and reduces outliers in the positive samples. We propose a new error measurement called fConsensus which exploits the parallel consensus to better define the concepts of exactness (mislabels) and completeness (missed labels) for prediction models. We present a series of experiments using the MultiLis Corpus where three listeners were tricked into believing that they had a one-on-one conversation with a speaker, while in fact they were recorded in parallel in interaction with the same speaker. In this paper we show that using parallel listener consensus can improve learning performance and represent better evaluation criteria for predictive models.
{"title":"Learning and evaluating response prediction models using parallel listener consensus","authors":"I. D. Kok, Derya Ozkan, D. Heylen, Louis-Philippe Morency","doi":"10.1145/1891903.1891908","DOIUrl":"https://doi.org/10.1145/1891903.1891908","url":null,"abstract":"Traditionally listener response prediction models are learned from pre-recorded dyadic interactions. Because of individual differences in behavior, these recordings do not capture the complete ground truth. Where the recorded listener did not respond to an opportunity provided by the speaker, another listener would have responded or vice versa. In this paper, we introduce the concept of parallel listener consensus where the listener responses from multiple parallel interactions are combined to better capture differences and similarities between individuals. We show how parallel listener consensus can be used for both learning and evaluating probabilistic prediction models of listener responses. To improve the learning performance, the parallel consensus helps identifying better negative samples and reduces outliers in the positive samples. We propose a new error measurement called fConsensus which exploits the parallel consensus to better define the concepts of exactness (mislabels) and completeness (missed labels) for prediction models. We present a series of experiments using the MultiLis Corpus where three listeners were tricked into believing that they had a one-on-one conversation with a speaker, while in fact they were recorded in parallel in interaction with the same speaker. In this paper we show that using parallel listener consensus can improve learning performance and represent better evaluation criteria for predictive models.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132931055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sergio Escalera, P. Radeva, Jordi Vitrià, Xavier Baró, B. Raducanu
Social network analysis became a common technique used to model and quantify the properties of social interactions. In this paper, we propose an integrated framework to explore the characteristics of a social network extracted from multimodal dyadic interactions. First, speech detection is performed through an audio/visual fusion scheme based on stacked sequential learning. In the audio domain, speech is detected through clusterization of audio features. Clusters are modelled by means of an One-state Hidden Markov Model containing a diagonal covariance Gaussian Mixture Model. In the visual domain, speech detection is performed through differential-based feature extraction from the segmented mouth region, and a dynamic programming matching procedure. Second, in order to model the dyadic interactions, we employed the Influence Model whose states encode the previous integrated audio/visual data. Third, the social network is extracted based on the estimated influences. For our study, we used a set of videos belonging to New York Times' Blogging Heads opinion blog. The results are reported both in terms of accuracy of the audio/visual data fusion and centrality measures used to characterize the social network.
{"title":"Modelling and analyzing multimodal dyadic interactions using social networks","authors":"Sergio Escalera, P. Radeva, Jordi Vitrià, Xavier Baró, B. Raducanu","doi":"10.1145/1891903.1891967","DOIUrl":"https://doi.org/10.1145/1891903.1891967","url":null,"abstract":"Social network analysis became a common technique used to model and quantify the properties of social interactions. In this paper, we propose an integrated framework to explore the characteristics of a social network extracted from multimodal dyadic interactions. First, speech detection is performed through an audio/visual fusion scheme based on stacked sequential learning. In the audio domain, speech is detected through clusterization of audio features. Clusters are modelled by means of an One-state Hidden Markov Model containing a diagonal covariance Gaussian Mixture Model. In the visual domain, speech detection is performed through differential-based feature extraction from the segmented mouth region, and a dynamic programming matching procedure. Second, in order to model the dyadic interactions, we employed the Influence Model whose states encode the previous integrated audio/visual data. Third, the social network is extracted based on the estimated influences. For our study, we used a set of videos belonging to New York Times' Blogging Heads opinion blog. The results are reported both in terms of accuracy of the audio/visual data fusion and centrality measures used to characterize the social network.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"53 1-2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116590650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Speak4itSM is a consumer-oriented mobile search application that leverages multimodal input and output to allow users to search for and act on local business information. It supports true multimodal integration where user inputs can be distributed over multiple input modes. In addition to specifying queries by voice (e.g., "bike repair shops near the golden gate bridge") users can combine speech and gesture. For example, "gas stations" + will return the gas stations along the specified route traced on the display. We provide interactive demonstrations of Speak4it on both the iPhone and iPad platforms and explain the underlying multimodal architecture and challenges of supporting multimodal interaction as a deployed mobile service.
{"title":"Speak4it: multimodal interaction for local search","authors":"Patrick Ehlen, Michael Johnston","doi":"10.1145/1891903.1891917","DOIUrl":"https://doi.org/10.1145/1891903.1891917","url":null,"abstract":"Speak4itSM is a consumer-oriented mobile search application that leverages multimodal input and output to allow users to search for and act on local business information. It supports true multimodal integration where user inputs can be distributed over multiple input modes. In addition to specifying queries by voice (e.g., \"bike repair shops near the golden gate bridge\") users can combine speech and gesture. For example, \"gas stations\" + <route drawn on display> will return the gas stations along the specified route traced on the display. We provide interactive demonstrations of Speak4it on both the iPhone and iPad platforms and explain the underlying multimodal architecture and challenges of supporting multimodal interaction as a deployed mobile service.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121982464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Matthew J. Pitts, G. Burnett, M. Williams, Tom Wellings
Touchscreens are increasingly being used in mobile devices and in-vehicle systems. While the usability benefits of touchscreens are acknowledged, their use places significant visual demand on the user due to the lack of tactile and kinaesthetic feedback. Haptic feedback is shown to improve performance in mobile devices, but little objective data is available regarding touchscreen feedback in an automotive scenario. A study was conducted to investigate the effects of visual and haptic touchscreen feedback on driver visual behaviour and driving performance using a simulated driving environment. Results showed a significant interaction between visual and haptic feedback, with the presence of haptic feedback compensating for changes in visual feedback. Driving performance was unaffected by feedback condition but degraded from a baseline measure when touchscreen tasks were introduced. Subjective responses indicated an improved user experience and increased confidence when haptic feedback was enabled.
{"title":"Does haptic feedback change the way we view touchscreens in cars?","authors":"Matthew J. Pitts, G. Burnett, M. Williams, Tom Wellings","doi":"10.1145/1891903.1891952","DOIUrl":"https://doi.org/10.1145/1891903.1891952","url":null,"abstract":"Touchscreens are increasingly being used in mobile devices and in-vehicle systems. While the usability benefits of touchscreens are acknowledged, their use places significant visual demand on the user due to the lack of tactile and kinaesthetic feedback. Haptic feedback is shown to improve performance in mobile devices, but little objective data is available regarding touchscreen feedback in an automotive scenario. A study was conducted to investigate the effects of visual and haptic touchscreen feedback on driver visual behaviour and driving performance using a simulated driving environment. Results showed a significant interaction between visual and haptic feedback, with the presence of haptic feedback compensating for changes in visual feedback. Driving performance was unaffected by feedback condition but degraded from a baseline measure when touchscreen tasks were introduced. Subjective responses indicated an improved user experience and increased confidence when haptic feedback was enabled.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"132 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133216007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Endowing embodied conversational agent with personality affords more natural modalities for their interaction with human interlocutors. To bridge the personality gap between users and agents, we designed minimal two personalities for corresponding agents i.e. an introverted and an extroverted agent. Each features a combination of different verbal and non-verbal behaviors. In this paper, we present an examination of the effects of the speaking and behavior styles of the two agents and explore the resulting design factors pertinent for spoken dialogue systems. The results indicate that users prefer the extroverted agent to the introverted one. The personality traits of the agents influence the users' preferences, dialogues, and behavior. Statistically, it is highly significant that users are more talkative with the extroverted agent. We also investigate the spontaneous speech disfluency of the dialogues and demonstrate that the extroverted behavior model reduce the user's speech disfluency. Furthermore, users having different mental models behave differently with the agents. The results and findings show that the minimal personalities of agents maximally influence the interlocutors' behaviors.
{"title":"Behavior and preference in minimal personality: a study on embodied conversational agents","authors":"Yuting Chen, A. Naveed, R. Porzel","doi":"10.1145/1891903.1891963","DOIUrl":"https://doi.org/10.1145/1891903.1891963","url":null,"abstract":"Endowing embodied conversational agent with personality affords more natural modalities for their interaction with human interlocutors. To bridge the personality gap between users and agents, we designed minimal two personalities for corresponding agents i.e. an introverted and an extroverted agent. Each features a combination of different verbal and non-verbal behaviors. In this paper, we present an examination of the effects of the speaking and behavior styles of the two agents and explore the resulting design factors pertinent for spoken dialogue systems. The results indicate that users prefer the extroverted agent to the introverted one. The personality traits of the agents influence the users' preferences, dialogues, and behavior. Statistically, it is highly significant that users are more talkative with the extroverted agent. We also investigate the spontaneous speech disfluency of the dialogues and demonstrate that the extroverted behavior model reduce the user's speech disfluency. Furthermore, users having different mental models behave differently with the agents. The results and findings show that the minimal personalities of agents maximally influence the interlocutors' behaviors.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133590184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
As the popularity and diversification of both Internet and its access devices, users' browsing experience of web pages is in great need of improvement. Traditional browsing mode of web elements such as table and image is passive, which limits users' browsing efficiency of web pages. In this paper, we propose to enhance browsing experience of table and image elements in web pages by enabling real-time interactive access to web tables and images. We design new browsing modes that help users improve their browsing efficiency including operation mode, record mode for web tables and normal mode, starred mode, advanced mode for web images. We design and implement a plug-in for Microsoft Internet Explorer, called iWebWidget, which provides a customized user interface supporting real-time interactive access to web tables and images. Besides, we carry out a user study to testify the usefulness of iWebWidget. Experimental results show that users are satisfied and really enjoy the new browsing modes for both web tables and images.
随着互联网及其接入设备的普及和多样化，用户的网页浏览体验亟待改善。传统的表格、图片等网页元素的浏览方式是被动的，限制了用户对网页的浏览效率。在本文中，我们建议通过实现对网页表格和图像的实时交互访问来增强网页中表格和图像元素的浏览体验。我们设计了新的浏览模式，帮助用户提高浏览效率，包括对网页表格的操作模式、记录模式和对网页图片的正常模式、星号模式、高级模式。我们为Microsoft Internet Explorer设计并实现了一个名为iWebWidget的插件，它提供了一个定制的用户界面，支持对web表和图像的实时交互访问。此外，我们进行了一个用户研究来证明iWebWidget的有用性。实验结果表明，用户满意并真正喜欢新的网页表格和图片浏览模式。
{"title":"Enhancing browsing experience of table and image elements in web pages","authors":"Wenchang Xu, Xin Yang, Yuanchun Shi","doi":"10.1145/1891903.1891935","DOIUrl":"https://doi.org/10.1145/1891903.1891935","url":null,"abstract":"As the popularity and diversification of both Internet and its access devices, users' browsing experience of web pages is in great need of improvement. Traditional browsing mode of web elements such as table and image is passive, which limits users' browsing efficiency of web pages. In this paper, we propose to enhance browsing experience of table and image elements in web pages by enabling real-time interactive access to web tables and images. We design new browsing modes that help users improve their browsing efficiency including operation mode, record mode for web tables and normal mode, starred mode, advanced mode for web images. We design and implement a plug-in for Microsoft Internet Explorer, called iWebWidget, which provides a customized user interface supporting real-time interactive access to web tables and images. Besides, we carry out a user study to testify the usefulness of iWebWidget. Experimental results show that users are satisfied and really enjoy the new browsing modes for both web tables and images.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"439 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125769860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sign language recognition (SLR) not only facilitates the communication between the deaf and hearing society, but also serves as a good basis for the development of gesture-based human-computer interaction (HCI). In this paper, the portable input devices based on accelerometers and surface electromyography (EMG) sensors worn on the forearm are presented, and an effective fusion strategy for combination of multi-sensor and multi-channel information is proposed to automatically recognize sign language at the subword classification level. Experimental results on the recognition of 121 frequently used Chinese sign language subwords demonstrate the feasibility of developing SLR system based on the presented portable input devices and that our proposed information fusion method is effective for automatic SLR. Our study will promote the realization of practical sign language recognizer and multimodal human-computer interfaces.
{"title":"Automatic recognition of sign language subwords based on portable accelerometer and EMG sensors","authors":"Yun Li, Xiang Chen, Jianxun Tian, Xu Zhang, Kongqiao Wang, Jihai Yang","doi":"10.1145/1891903.1891926","DOIUrl":"https://doi.org/10.1145/1891903.1891926","url":null,"abstract":"Sign language recognition (SLR) not only facilitates the communication between the deaf and hearing society, but also serves as a good basis for the development of gesture-based human-computer interaction (HCI). In this paper, the portable input devices based on accelerometers and surface electromyography (EMG) sensors worn on the forearm are presented, and an effective fusion strategy for combination of multi-sensor and multi-channel information is proposed to automatically recognize sign language at the subword classification level. Experimental results on the recognition of 121 frequently used Chinese sign language subwords demonstrate the feasibility of developing SLR system based on the presented portable input devices and that our proposed information fusion method is effective for automatic SLR. Our study will promote the realization of practical sign language recognizer and multimodal human-computer interfaces.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114464145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：