The use of dual eye-tracking is investigated in a collaborative game setting. Social context influences individual gaze and action during a collaborative Tetris game: results show that experts as well as novices adapt their playing style when interacting in mixed ability pairs. The long term goal of our work is to design adaptive gaze awareness tools that take the pair composition into account. We therefore investigate the automatic detection (or recognition) of pair composition using dual gaze-based as well as action-based multimodal features. We describe several methods for the improvement of detection (or recognition) and experimentally demonstrate their effectiveness, especially in the situations when the collected gaze data are noisy.
{"title":"Gaze quality assisted automatic recognition of social contexts in collaborative Tetris","authors":"Weifeng Li, Marc-Antoine Nüssli, Patrick Jermann","doi":"10.1145/1891903.1891914","DOIUrl":"https://doi.org/10.1145/1891903.1891914","url":null,"abstract":"The use of dual eye-tracking is investigated in a collaborative game setting. Social context influences individual gaze and action during a collaborative Tetris game: results show that experts as well as novices adapt their playing style when interacting in mixed ability pairs. The long term goal of our work is to design adaptive gaze awareness tools that take the pair composition into account. We therefore investigate the automatic detection (or recognition) of pair composition using dual gaze-based as well as action-based multimodal features. We describe several methods for the improvement of detection (or recognition) and experimentally demonstrate their effectiveness, especially in the situations when the collected gaze data are noisy.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117165874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dairazalia Sanchez-Cortes, O. Aran, M. S. Mast, D. Gática-Pérez
This paper addresses firstly an analysis on how an emergent leader is perceived in newly formed small-groups, and secondly, explore correlations between perception of leadership and automatically extracted nonverbal communicative cues. We hypothesize that the difference in individual nonverbal features between emergent leaders and non-emergent leaders is significant and measurable using speech activity. Our results on a new interaction corpus show that such an approach is promising, identifying the emergent leader with an accuracy of up to 80%.
{"title":"Identifying emergent leadership in small groups using nonverbal communicative cues","authors":"Dairazalia Sanchez-Cortes, O. Aran, M. S. Mast, D. Gática-Pérez","doi":"10.1145/1891903.1891953","DOIUrl":"https://doi.org/10.1145/1891903.1891953","url":null,"abstract":"This paper addresses firstly an analysis on how an emergent leader is perceived in newly formed small-groups, and secondly, explore correlations between perception of leadership and automatically extracted nonverbal communicative cues. We hypothesize that the difference in individual nonverbal features between emergent leaders and non-emergent leaders is significant and measurable using speech activity. Our results on a new interaction corpus show that such an approach is promising, identifying the emergent leader with an accuracy of up to 80%.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128142537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
People's activities around their photo collections are often highly dynamic and unstructured, such as casual browsing and searching or loosely structured storytelling. User interfaces to support such an exploratory behavior are a challenging research question. We explore ways to enhance the flexibility in dealing with photo collections and designed a system named PhotoMagnets. It uses a magnet metaphor in addition to more traditional interface elements in order to support a flexible combination of structured and unstructured photo browsing and searching. In an evaluation we received positive feedback especially on the flexibility provided by this approach.
{"title":"PhotoMagnets: supporting flexible browsing and searching in photo collections","authors":"Ya-Xi Chen, M. Reiter, A. Butz","doi":"10.1145/1891903.1891936","DOIUrl":"https://doi.org/10.1145/1891903.1891936","url":null,"abstract":"People's activities around their photo collections are often highly dynamic and unstructured, such as casual browsing and searching or loosely structured storytelling. User interfaces to support such an exploratory behavior are a challenging research question. We explore ways to enhance the flexibility in dealing with photo collections and designed a system named PhotoMagnets. It uses a magnet metaphor in addition to more traditional interface elements in order to support a flexible combination of structured and unstructured photo browsing and searching. In an evaluation we received positive feedback especially on the flexibility provided by this approach.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131203196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Speech recognition systems are not typically able to produce error-free results in real scenarios. On account of this, human intervention is usually needed. This intervention can be included into the system by following the Computer Assisted Speech Transcription (CAST) approach, where the user constantly interacts with the system during the transcription process. In order to improve this user interaction, a speech multi-modal interface is proposed here. In addition, the user of word graphs within CAST aimed at facilitating the design of such interface as well as improving the system response time is also discussed
{"title":"Multi-modal computer assisted speech transcription","authors":"Luis Rodríguez, I. García-Varea, E. Vidal","doi":"10.1145/1891903.1891943","DOIUrl":"https://doi.org/10.1145/1891903.1891943","url":null,"abstract":"Speech recognition systems are not typically able to produce error-free results in real scenarios. On account of this, human intervention is usually needed. This intervention can be included into the system by following the Computer Assisted Speech Transcription (CAST) approach, where the user constantly interacts with the system during the transcription process. In order to improve this user interaction, a speech multi-modal interface is proposed here. In addition, the user of word graphs within CAST aimed at facilitating the design of such interface as well as improving the system response time is also discussed","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132283466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nicolás Serrano, Adrià Giménez, A. Sanchís, Alfons Juan-Císcar
Active learning strategies are being increasingly used in a variety of real-world tasks, though their application to handwritten text transcription in old manuscripts remains nearly unexplored. The basic idea is to follow a sequential, line-byline transcription of the whole manuscript in which a continuously retrained system interacts with the user to efficiently transcribe each new line. This approach has been recently explored using a conventional strategy by which the user is only asked to supervise words that are not recognized with high confidence. In this paper, the conventional strategy is improved by also letting the system to recompute most probable hypotheses with the constraints imposed by user supervisions. In particular, two strategies are studied which differ in the frequency of hypothesis recomputation on the current line: after each (iterative) or all (delayed) user corrections. Empirical results are reported on two real tasks showing that these strategies outperform the conventional approach.
{"title":"Active learning strategies for handwritten text transcription","authors":"Nicolás Serrano, Adrià Giménez, A. Sanchís, Alfons Juan-Císcar","doi":"10.1145/1891903.1891962","DOIUrl":"https://doi.org/10.1145/1891903.1891962","url":null,"abstract":"Active learning strategies are being increasingly used in a variety of real-world tasks, though their application to handwritten text transcription in old manuscripts remains nearly unexplored. The basic idea is to follow a sequential, line-byline transcription of the whole manuscript in which a continuously retrained system interacts with the user to efficiently transcribe each new line. This approach has been recently explored using a conventional strategy by which the user is only asked to supervise words that are not recognized with high confidence. In this paper, the conventional strategy is improved by also letting the system to recompute most probable hypotheses with the constraints imposed by user supervisions. In particular, two strategies are studied which differ in the frequency of hypothesis recomputation on the current line: after each (iterative) or all (delayed) user corrections. Empirical results are reported on two real tasks showing that these strategies outperform the conventional approach.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133185277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The availability of numerous networked interaction devices within smart environments makes the exploitation of these devices for innovative and more natural interaction possible. In our work we make use of TVs with remote controls, picture frames, mobile phones, touch screens, stereos and PCs to create multimodal user interfaces. The combination of the interaction capabilities of the different devices allows to achieve a more suitable interaction for a situation. Changing situations can then require the dynamic redistribution of the created interfaces and the alteration of the used modalities and devices to keep up the interaction. In this paper we describe our approach for dynamically (re-) distributing user interfaces at run-time. A distribution component is responsible for determining the devices for the interaction based on the (changing) environment situation and the user interface requirements. The component provides possibilities to the application developer and to the user to influence the distribution according to their needs. A user interface model describes the interaction and the modality relations according to the CARE properties (Complementarity, Assignment, Redundancy and Equivalency) and a context model gathers and provides information about the environment.
{"title":"Dynamic user interface distribution for flexible multimodal interaction","authors":"Marco Blumendorf, Dirk Roscher, S. Albayrak","doi":"10.1145/1891903.1891930","DOIUrl":"https://doi.org/10.1145/1891903.1891930","url":null,"abstract":"The availability of numerous networked interaction devices within smart environments makes the exploitation of these devices for innovative and more natural interaction possible. In our work we make use of TVs with remote controls, picture frames, mobile phones, touch screens, stereos and PCs to create multimodal user interfaces. The combination of the interaction capabilities of the different devices allows to achieve a more suitable interaction for a situation. Changing situations can then require the dynamic redistribution of the created interfaces and the alteration of the used modalities and devices to keep up the interaction. In this paper we describe our approach for dynamically (re-) distributing user interfaces at run-time. A distribution component is responsible for determining the devices for the interaction based on the (changing) environment situation and the user interface requirements. The component provides possibilities to the application developer and to the user to influence the distribution according to their needs. A user interface model describes the interaction and the modality relations according to the CARE properties (Complementarity, Assignment, Redundancy and Equivalency) and a context model gathers and provides information about the environment.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129942058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper reports a new intramodal haptic illusion. This illusion involves a person pressing on a rigid surface and perceiving that the surface is compliant, i.e. perceiving that the contact point displaces into the surface. The design process, method and conditions used to create this illusion are described in detail. A user study is also reported in which all participants using variants of the basic method experienced the illusion, demonstrating the effectiveness of the method. This study also offers an initial indication of the mechanical dimensions of illusory compliance that could be manipulated by varying the stimuli presented to the users. This method could be used to augment touch interaction with mobile devices, transcending the rigid two-dimensional tangible surface (touch display) currently found on them.
{"title":"3D-press: haptic illusion of compliance when pressing on a rigid surface","authors":"Johan Kildal","doi":"10.1145/1891903.1891931","DOIUrl":"https://doi.org/10.1145/1891903.1891931","url":null,"abstract":"This paper reports a new intramodal haptic illusion. This illusion involves a person pressing on a rigid surface and perceiving that the surface is compliant, i.e. perceiving that the contact point displaces into the surface. The design process, method and conditions used to create this illusion are described in detail. A user study is also reported in which all participants using variants of the basic method experienced the illusion, demonstrating the effectiveness of the method. This study also offers an initial indication of the mechanical dimensions of illusory compliance that could be manipulated by varying the stimuli presented to the users. This method could be used to augment touch interaction with mobile devices, transcending the rigid two-dimensional tangible surface (touch display) currently found on them.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132017596","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Ambient Spotlight is a prototype system based on personal meeting capture using a laptop and a portable microphone array. The system automatically recognises and structures the meeting content using automatic speech recognition, topic segmentation and extractive summarisation. The recognised speech in the meeting is used to construct queries to automatically link meeting segments to other relevant material, both multimodal and textual. The interface to the system is constructed around a standard calendar interface, and it is integrated with the laptop's standard indexing, search and retrieval.
{"title":"The Ambient Spotlight: personal multimodal search without query","authors":"J. Kilgour, J. Carletta, S. Renals","doi":"10.1145/1891903.1891919","DOIUrl":"https://doi.org/10.1145/1891903.1891919","url":null,"abstract":"The Ambient Spotlight is a prototype system based on personal meeting capture using a laptop and a portable microphone array. The system automatically recognises and structures the meeting content using automatic speech recognition, topic segmentation and extractive summarisation. The recognised speech in the meeting is used to construct queries to automatically link meeting segments to other relevant material, both multimodal and textual. The interface to the system is constructed around a standard calendar interface, and it is integrated with the laptop's standard indexing, search and retrieval.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114207473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kaiming Li, Lei Guo, C. Faraco, Dajiang Zhu, Fan Deng, Tuo Zhang, Xi Jiang, Degang Zhang, Hanbo Chen, Xintao Hu, L. Miller, Tianming Liu
A variety of user attention models for video/audio streams have been developed for video summarization and abstraction, in order to facilitate efficient video browsing and indexing. Essentially, human brain is the end user and evaluator of multimedia content and representation, and its responses can provide meaningful guidelines for multimedia stream summarization. For example, video/audio segments that significantly activate the visual, auditory, language and working memory systems of the human brain should be considered more important than others. It should be noted that user experience studies could be useful for such evaluations, but are suboptimal in terms of their capability of accurately capturing the full-length dynamics and interactions of the brain's response. This paper presents our preliminary efforts in applying the brain imaging technique of functional magnetic resonance imaging (fMRI) to quantify and model the dynamics and interactions between multimedia streams and brain response, when the human subjects are presented with the multimedia clips, in order to develop human-centered attention models that can be used to guide and facilitate more effective and efficient multimedia summarization. Our initial results are encouraging.
{"title":"Human-centered attention models for video summarization","authors":"Kaiming Li, Lei Guo, C. Faraco, Dajiang Zhu, Fan Deng, Tuo Zhang, Xi Jiang, Degang Zhang, Hanbo Chen, Xintao Hu, L. Miller, Tianming Liu","doi":"10.1145/1891903.1891938","DOIUrl":"https://doi.org/10.1145/1891903.1891938","url":null,"abstract":"A variety of user attention models for video/audio streams have been developed for video summarization and abstraction, in order to facilitate efficient video browsing and indexing. Essentially, human brain is the end user and evaluator of multimedia content and representation, and its responses can provide meaningful guidelines for multimedia stream summarization. For example, video/audio segments that significantly activate the visual, auditory, language and working memory systems of the human brain should be considered more important than others. It should be noted that user experience studies could be useful for such evaluations, but are suboptimal in terms of their capability of accurately capturing the full-length dynamics and interactions of the brain's response. This paper presents our preliminary efforts in applying the brain imaging technique of functional magnetic resonance imaging (fMRI) to quantify and model the dynamics and interactions between multimedia streams and brain response, when the human subjects are presented with the multimedia clips, in order to develop human-centered attention models that can be used to guide and facilitate more effective and efficient multimedia summarization. Our initial results are encouraging.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133601826","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multimodal interaction is becoming common in many kinds of devices, particularly mobile phones. If care is not taken in design and implementation, there may be latencies in the timing of feedback in the different modalities may have unintended effects on users. This paper introduces an easy to implement multimodal latency measurement tool for touchscreen interaction. It uses off-the-shelf components and free software and is capable of measuring latencies accurately between different interaction events in different modalities. The tool uses a high-speed camera, a mirror, a microphone and an accelerometer to measure the touch, visual, audio and tactile feedback events that occur in touchscreen interaction. The microphone and the accelerometer are both interfaced with a standard PC soundcard that makes the measurement and analysis simple. The latencies are obtained by hand and eye using a slow-motion video player and an audio editor. To validate the tool, we measured four commercial mobile phones. Our results show that there are significant differences in latencies, not only between the devices, but also between different applications and modalities within one device. In this paper the focus is on mobile touchscreen devices, but with minor modifications our tool could be also used in other domains.
{"title":"Feedback is... late: measuring multimodal delays in mobile device touchscreen interaction","authors":"Topi Kaaresoja, S. Brewster","doi":"10.1145/1891903.1891907","DOIUrl":"https://doi.org/10.1145/1891903.1891907","url":null,"abstract":"Multimodal interaction is becoming common in many kinds of devices, particularly mobile phones. If care is not taken in design and implementation, there may be latencies in the timing of feedback in the different modalities may have unintended effects on users. This paper introduces an easy to implement multimodal latency measurement tool for touchscreen interaction. It uses off-the-shelf components and free software and is capable of measuring latencies accurately between different interaction events in different modalities. The tool uses a high-speed camera, a mirror, a microphone and an accelerometer to measure the touch, visual, audio and tactile feedback events that occur in touchscreen interaction. The microphone and the accelerometer are both interfaced with a standard PC soundcard that makes the measurement and analysis simple. The latencies are obtained by hand and eye using a slow-motion video player and an audio editor. To validate the tool, we measured four commercial mobile phones. Our results show that there are significant differences in latencies, not only between the devices, but also between different applications and modalities within one device. In this paper the focus is on mobile touchscreen devices, but with minor modifications our tool could be also used in other domains.","PeriodicalId":181145,"journal":{"name":"ICMI-MLMI '10","volume":"26 12","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114129927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：