In some circumstances, immersion in virtual environments with the aid of virtual reality (VR) equipment can create feelings of anxiety in users and be experienced as something “frightening”, “oppressive”, “alienating”, “dehumanizing”, or “dystopian”. Sometimes (e.g., in exposure therapy or VR gaming), a virtual environment is intended to have such psychological impacts on users; however, such effects can also arise unintentionally due to the environment’s poor architectural design. Designers of virtual environments may employ user-centered design (UCD) to incrementally improve a design and generate a user experience more closely resembling the type desired; however, UCD can yield suboptimal results if an initial design relied on an inappropriate architectural approach. This study developed a framework that can facilitate the purposeful selection of the most appropriate architectural approach by drawing on Norberg-Schulz’s established phenomenological account of real-world architectural modes. By considering the unique possibilities for structuring and experiencing space within virtual environments and reinterpreting Norberg-Schulz’s schemas in the context of virtual environment design, a novel framework was formulated that explicates six fundamental “architectural paradigms” available to designers of virtual environments. It was shown that the application of this framework could easily be incorporated as an additional step within the UCD process.
{"title":"A Phenomenological Framework of Architectural Paradigms for the User-Centered Design of Virtual Environments","authors":"Matthew E. Gladden","doi":"10.3390/MTI2040080","DOIUrl":"https://doi.org/10.3390/MTI2040080","url":null,"abstract":"In some circumstances, immersion in virtual environments with the aid of virtual reality (VR) equipment can create feelings of anxiety in users and be experienced as something “frightening”, “oppressive”, “alienating”, “dehumanizing”, or “dystopian”. Sometimes (e.g., in exposure therapy or VR gaming), a virtual environment is intended to have such psychological impacts on users; however, such effects can also arise unintentionally due to the environment’s poor architectural design. Designers of virtual environments may employ user-centered design (UCD) to incrementally improve a design and generate a user experience more closely resembling the type desired; however, UCD can yield suboptimal results if an initial design relied on an inappropriate architectural approach. This study developed a framework that can facilitate the purposeful selection of the most appropriate architectural approach by drawing on Norberg-Schulz’s established phenomenological account of real-world architectural modes. By considering the unique possibilities for structuring and experiencing space within virtual environments and reinterpreting Norberg-Schulz’s schemas in the context of virtual environment design, a novel framework was formulated that explicates six fundamental “architectural paradigms” available to designers of virtual environments. It was shown that the application of this framework could easily be incorporated as an additional step within the UCD process.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040080","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756300","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, tangible user interfaces (TUI) have gained in popularity in educational contexts, among others to implement problem-solving and discovery learning science activities. In the context of an interdisciplinary and cross-institutional collaboration, we conducted a multimodal EMCA-based video user study involving a TUI-mediated bicycle mechanics simulation. This article focusses on the discovering work of a group of three students with regard to a particular tangible object (a red button), designed to support participants engagement with the underlying physics aspects and its consequences with regard to their engagement with the targeted mechanics aspects.
{"title":"\"There Was No Green Tick\": Discovering the Functions of a Widget in a Joint Problem-Solving Activity and the Consequences for the Participants' Discovering Process","authors":"Patrick Sunnen, Béatrice Arend, V. Maquil","doi":"10.3390/MTI2040076","DOIUrl":"https://doi.org/10.3390/MTI2040076","url":null,"abstract":"In recent years, tangible user interfaces (TUI) have gained in popularity in educational contexts, among others to implement problem-solving and discovery learning science activities. In the context of an interdisciplinary and cross-institutional collaboration, we conducted a multimodal EMCA-based video user study involving a TUI-mediated bicycle mechanics simulation. This article focusses on the discovering work of a group of three students with regard to a particular tangible object (a red button), designed to support participants engagement with the underlying physics aspects and its consequences with regard to their engagement with the targeted mechanics aspects.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040076","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we present the results of an online survey (N = 170) on emotional situations on the road. In particular, we asked potential early adopters to remember a situation where they felt either an intense positive or negative emotion while driving. Our research is motivated by imminent disruptions in the automotive sector due to automated driving and the accompanying switch to selling driving experiences over horsepower. This creates a need to focus on the driver’s emotion when designing in-car interfaces. As a result of our research, we present a set of propositions for affective car interfaces based on real-life experiences. With our work we aim to support the design of affective car interfaces and give designers a foundation to build upon. We find respondents often connect positive emotions with enjoying their independence, while negative experiences are associated mostly with traffic behavior. Participants who experienced negative situations wished for better information management and a higher degree of automation. Drivers with positive emotions generally wanted to experience the situation more genuinely, for example, by switching to a “back-to-basic” mode. We explore these statements and discuss recommendations for the design of affective interfaces in future cars.
{"title":"A Survey to Understand Emotional Situations on the Road and What They Mean for Affective Automotive UIs","authors":"Michael Braun, Bastian Pfleging, Florian Alt","doi":"10.3390/MTI2040075","DOIUrl":"https://doi.org/10.3390/MTI2040075","url":null,"abstract":"In this paper, we present the results of an online survey (N = 170) on emotional situations on the road. In particular, we asked potential early adopters to remember a situation where they felt either an intense positive or negative emotion while driving. Our research is motivated by imminent disruptions in the automotive sector due to automated driving and the accompanying switch to selling driving experiences over horsepower. This creates a need to focus on the driver’s emotion when designing in-car interfaces. As a result of our research, we present a set of propositions for affective car interfaces based on real-life experiences. With our work we aim to support the design of affective car interfaces and give designers a foundation to build upon. We find respondents often connect positive emotions with enjoying their independence, while negative experiences are associated mostly with traffic behavior. Participants who experienced negative situations wished for better information management and a higher degree of automation. Drivers with positive emotions generally wanted to experience the situation more genuinely, for example, by switching to a “back-to-basic” mode. We explore these statements and discuss recommendations for the design of affective interfaces in future cars.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040075","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Social allostasis is a mechanism of adaptation that permits individuals to dynamically adapt their physiology to changing physical and social conditions. Oxytocin (OT) is widely considered to be one of the hormones that drives and adapts social behaviours. While its precise effects remain unclear, two areas where OT may promote adaptation are by affecting social salience, and affecting internal responses of performing social behaviours. Working towards a model of dynamic adaptation through social allostasis in simulated embodied agents, and extending our previous work studying OT-inspired modulation of social salience, we present a model and experiments that investigate the effects and adaptive value of allostatic processes based on hormonal (OT) modulation of affective elements of a social behaviour. In particular, we investigate and test the effects and adaptive value of modulating the degree of satisfaction of tactile contact in a social motivation context in a small simulated agent society across different environmental challenges (related to availability of food) and effects of OT modulation of social salience as a motivational incentive. Our results show that the effects of these modulatory mechanisms have different (positive or negative) adaptive value across different groups and under different environmental circumstance in a way that supports the context-dependent nature of OT, put forward by the interactionist approach to OT modulation in biological agents. In terms of simulation models, this means that OT modulation of the mechanisms that we have described should be context-dependent in order to maximise viability of our socially adaptive agents, illustrating the relevance of social allostasis mechanisms.
{"title":"Modelling Adaptation through Social Allostasis: Modulating the Effects of Social Touch with Oxytocin in Embodied Agents","authors":"Imran Khan, Lola Cañamero","doi":"10.3390/MTI2040067","DOIUrl":"https://doi.org/10.3390/MTI2040067","url":null,"abstract":"Social allostasis is a mechanism of adaptation that permits individuals to dynamically adapt their physiology to changing physical and social conditions. Oxytocin (OT) is widely considered to be one of the hormones that drives and adapts social behaviours. While its precise effects remain unclear, two areas where OT may promote adaptation are by affecting social salience, and affecting internal responses of performing social behaviours. Working towards a model of dynamic adaptation through social allostasis in simulated embodied agents, and extending our previous work studying OT-inspired modulation of social salience, we present a model and experiments that investigate the effects and adaptive value of allostatic processes based on hormonal (OT) modulation of affective elements of a social behaviour. In particular, we investigate and test the effects and adaptive value of modulating the degree of satisfaction of tactile contact in a social motivation context in a small simulated agent society across different environmental challenges (related to availability of food) and effects of OT modulation of social salience as a motivational incentive. Our results show that the effects of these modulatory mechanisms have different (positive or negative) adaptive value across different groups and under different environmental circumstance in a way that supports the context-dependent nature of OT, put forward by the interactionist approach to OT modulation in biological agents. In terms of simulation models, this means that OT modulation of the mechanisms that we have described should be context-dependent in order to maximise viability of our socially adaptive agents, illustrating the relevance of social allostasis mechanisms.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040067","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This research investigates the effect of different user characteristics on the perception and evaluation of an agent’s appearance variables. Therefore, two different experiments have been conducted. In a 3 × 3 × 5 within-subjects design (Study 1; N = 59), three different target groups (students, elderly, and cognitively impaired people) evaluated 30 different agent appearances that varied in species (human, animal, and robot) and realism (high detail, low detail, stylized shades, stylized proportion, and stylized shade with stylized proportion). Study 2 (N = 792) focused on the effect of moderating variables regarding the same appearance variables and aims to supplement findings of Study 1 based on a 3 × 5 between-subjects design. Results showed effects of species and realism on person perception, users’ liking, and using intention. In a direct comparison, a higher degree of realism was perceived as more positive, while those effects were not replicated in Study 2. Further on, a majority evaluated nonhumanoid agents more positively. Since no interaction effects of species and realism have been found, the effects of stylization seem to equally influence the perception for all kind of species. Moreover, the importance of the target group’s preference was demonstrated, since differences with regard to the appearance evaluation were found.
{"title":"A Two-Study Approach to Explore the Effect of User Characteristics on Users' Perception and Evaluation of a Virtual Assistant's Appearance","authors":"Carolin Straßmann, N. Krämer","doi":"10.3390/MTI2040066","DOIUrl":"https://doi.org/10.3390/MTI2040066","url":null,"abstract":"This research investigates the effect of different user characteristics on the perception and evaluation of an agent’s appearance variables. Therefore, two different experiments have been conducted. In a 3 × 3 × 5 within-subjects design (Study 1; N = 59), three different target groups (students, elderly, and cognitively impaired people) evaluated 30 different agent appearances that varied in species (human, animal, and robot) and realism (high detail, low detail, stylized shades, stylized proportion, and stylized shade with stylized proportion). Study 2 (N = 792) focused on the effect of moderating variables regarding the same appearance variables and aims to supplement findings of Study 1 based on a 3 × 5 between-subjects design. Results showed effects of species and realism on person perception, users’ liking, and using intention. In a direct comparison, a higher degree of realism was perceived as more positive, while those effects were not replicated in Study 2. Further on, a majority evaluated nonhumanoid agents more positively. Since no interaction effects of species and realism have been found, the effects of stylization seem to equally influence the perception for all kind of species. Moreover, the importance of the target group’s preference was demonstrated, since differences with regard to the appearance evaluation were found.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040066","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The context of the work presented in this article is the assessment and automated evaluation of human behaviour. To facilitate this, a formalism is presented which is unambiguous as well as such that it can be implemented and interpreted in an automated manner. In the greater scheme of things, comparable behaviour evaluation requires comparable assessment scenarios and, to this end, computer games are considered as controllable and abstract environments. Within this context, a model for behavioural AI is presented which was designed around the objectives of: (a) being able to play rationally; (b) adhering to formally stated behaviour preferences; and (c) ensuring that very specific circumstances can be forced to arise within a game. The presented work is based on established models from the field of behavioural psychology, formal logic as well as approaches from game theory and related fields. The suggested model for behavioural AI has been used to implement and test a game, as well as AI players that exhibit specific behavioural preferences. The overall aim of this article is to enable the readers to design their own AI implementation, using the formalisms and models they prefer and to a level of complexity they desire.
{"title":"Designing Behavioural Artificial Intelligence to Record, Assess and Evaluate Human Behaviour","authors":"H. Hildmann","doi":"10.3390/mti2040063","DOIUrl":"https://doi.org/10.3390/mti2040063","url":null,"abstract":"The context of the work presented in this article is the assessment and automated evaluation of human behaviour. To facilitate this, a formalism is presented which is unambiguous as well as such that it can be implemented and interpreted in an automated manner. In the greater scheme of things, comparable behaviour evaluation requires comparable assessment scenarios and, to this end, computer games are considered as controllable and abstract environments. Within this context, a model for behavioural AI is presented which was designed around the objectives of: (a) being able to play rationally; (b) adhering to formally stated behaviour preferences; and (c) ensuring that very specific circumstances can be forced to arise within a game. The presented work is based on established models from the field of behavioural psychology, formal logic as well as approaches from game theory and related fields. The suggested model for behavioural AI has been used to implement and test a game, as well as AI players that exhibit specific behavioural preferences. The overall aim of this article is to enable the readers to design their own AI implementation, using the formalisms and models they prefer and to a level of complexity they desire.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/mti2040063","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Most children can naturally engage in play and by this, develop skills while interacting with their peers and toys. However, children with Autism Spectrum Disorder (ASD) often show impairments in play skills which result in limited opportunities for interaction with others and the learning of social skills. In this regard, robotic devices/toys that can provide simple and attractive indications are advantageous to engage children with ASD in play activities that require social and interaction skills. This project proposes a new interaction method using paired robotic devices called COLOLO to facilitate a fundamental exchange of intention in communication so-called turn-taking. These tangible devices are designed to sense the user’s manipulation, send a message to the paired device, and display visual cues for assisting children to achieve turn-taking through play. On the sessions with COLOLO there are two devices, one held by the therapist and one by the child, and they take turns to manipulate the toys and change their colors. In this article, two experimental conditions or interaction rules: the “two-sided lighting rule” and the “one-sided lighting rule" were introduced. The two interactions rules differ from each on the way the devices used the visual cues to indicate the turn-holder. The effect of each interaction rule on children’s turn-taking behaviors was investigated through an experimental study with four children with ASD. From the results, we found that with the one-sided lighting rule participants tended to shift their gaze more and to decrease the failed attempts of turn-taking. The discussion covers the possibilities of using paired devices to describe participants’ behaviors related to turn-taking quantitatively.
{"title":"Effect of Sensory Feedback on Turn-Taking Using Paired Devices for Children with ASD","authors":"Eleuda Nuñez, Soichiro Matsuda, Masakazu Hirokawa, Jun-ichi Yamamoto, Kenji Suzuki","doi":"10.3390/MTI2040061","DOIUrl":"https://doi.org/10.3390/MTI2040061","url":null,"abstract":"Most children can naturally engage in play and by this, develop skills while interacting with their peers and toys. However, children with Autism Spectrum Disorder (ASD) often show impairments in play skills which result in limited opportunities for interaction with others and the learning of social skills. In this regard, robotic devices/toys that can provide simple and attractive indications are advantageous to engage children with ASD in play activities that require social and interaction skills. This project proposes a new interaction method using paired robotic devices called COLOLO to facilitate a fundamental exchange of intention in communication so-called turn-taking. These tangible devices are designed to sense the user’s manipulation, send a message to the paired device, and display visual cues for assisting children to achieve turn-taking through play. On the sessions with COLOLO there are two devices, one held by the therapist and one by the child, and they take turns to manipulate the toys and change their colors. In this article, two experimental conditions or interaction rules: the “two-sided lighting rule” and the “one-sided lighting rule\" were introduced. The two interactions rules differ from each on the way the devices used the visual cues to indicate the turn-holder. The effect of each interaction rule on children’s turn-taking behaviors was investigated through an experimental study with four children with ASD. From the results, we found that with the one-sided lighting rule participants tended to shift their gaze more and to decrease the failed attempts of turn-taking. The discussion covers the possibilities of using paired devices to describe participants’ behaviors related to turn-taking quantitatively.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2040061","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mario Neururer, Stephan Schlögl, Luisa Brinkschulte, Aleksander Groth
In 1950, Alan Turing proposed his concept of universal machines, emphasizing their abilities to learn, think, and behave in a human-like manner. Today, the existence of intelligent agents imitating human characteristics is more relevant than ever. They have expanded to numerous aspects of daily life. Yet, while they are often seen as work simplifiers, their interactions usually lack social competence. In particular, they miss what one may call authenticity. In the study presented in this paper, we explore how characteristics of social intelligence may enhance future agent implementations. Interviews and an open question survey with experts from different fields have led to a shared understanding of what it would take to make intelligent virtual agents, in particular messaging agents (i.e., chat bots), more authentic. Results suggest that showcasing a transparent purpose, learning from experience, anthropomorphizing, human-like conversational behavior, and coherence, are guiding characteristics for agent authenticity and should consequently allow for and support a better coexistence of artificial intelligence technology with its respective users.
{"title":"Perceptions on Authenticity in Chat Bots","authors":"Mario Neururer, Stephan Schlögl, Luisa Brinkschulte, Aleksander Groth","doi":"10.3390/MTI2030060","DOIUrl":"https://doi.org/10.3390/MTI2030060","url":null,"abstract":"In 1950, Alan Turing proposed his concept of universal machines, emphasizing their abilities to learn, think, and behave in a human-like manner. Today, the existence of intelligent agents imitating human characteristics is more relevant than ever. They have expanded to numerous aspects of daily life. Yet, while they are often seen as work simplifiers, their interactions usually lack social competence. In particular, they miss what one may call authenticity. In the study presented in this paper, we explore how characteristics of social intelligence may enhance future agent implementations. Interviews and an open question survey with experts from different fields have led to a shared understanding of what it would take to make intelligent virtual agents, in particular messaging agents (i.e., chat bots), more authentic. Results suggest that showcasing a transparent purpose, learning from experience, anthropomorphizing, human-like conversational behavior, and coherence, are guiding characteristics for agent authenticity and should consequently allow for and support a better coexistence of artificial intelligence technology with its respective users.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030060","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755935","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tele-guidance systems for the remote monitoring and maintenance of equipment have been extensively investigated. Such systems enable a remote helper to provide guidance to a local worker while perceiving local conditions. In this study, we propose a tele-guidance system that supports the anticipation of an interlocutor’s actions during communication. Our proposed system enables a helper and worker to anticipate each other’s actions by allowing them to move around in the workspace freely and observe each other’s non-verbal cues (e.g., body motions and other gestures) through a head-mounted display. We conducted an experiment to compare the effectiveness of our proposed method with that of existing methods (a simple tele-pointer) that support anticipation during communication.
{"title":"Tele-Guidance System to Support Anticipation during Communication","authors":"T. Yamamoto, M. Otsuki, H. Kuzuoka, Yusuke Suzuki","doi":"10.3390/MTI2030055","DOIUrl":"https://doi.org/10.3390/MTI2030055","url":null,"abstract":"Tele-guidance systems for the remote monitoring and maintenance of equipment have been extensively investigated. Such systems enable a remote helper to provide guidance to a local worker while perceiving local conditions. In this study, we propose a tele-guidance system that supports the anticipation of an interlocutor’s actions during communication. Our proposed system enables a helper and worker to anticipate each other’s actions by allowing them to move around in the workspace freely and observe each other’s non-verbal cues (e.g., body motions and other gestures) through a head-mounted display. We conducted an experiment to compare the effectiveness of our proposed method with that of existing methods (a simple tele-pointer) that support anticipation during communication.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030055","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tangible technologies are considered promising tools for learning, by enabling multimodal interaction through physical action and manipulation of physical and digital elements, thus facilitating representational concrete–abstract links. A key concept in a tangible system is that its physical components are objects of interest, with associated meanings relevant to the context. Tangible technologies are said to provide ‘natural’ mappings that employ spatial analogies and adhere to cultural standards, capitalising on people’s familiarity with the physical world. Students with intellectual disabilities particularly benefit from interaction with tangibles, given their difficulties with perception and abstraction. However, symbolic information does not always have an obvious physical equivalent, and meanings do not reside in the representations used in the artefacts themselves, but in the ways they are manipulated and interpreted. In educational contexts, meaning attached to artefacts by designers is not necessarily transparent to students, nor interpreted by them as the designer predicted. Using artefacts and understanding their significance is of utmost importance for the construction of knowledge within the learning process; hence the need to study the use of the artefacts in contexts of practice and how they are transformed by the students. This article discusses how children with intellectual disabilities conceptually interpreted the elements of four tangible artefacts, and which characteristics of these tangibles were key for productive, multimodal interaction, thus potentially guiding designers and educators. Analysis shows the importance of designing physical-digital semantic mappings that capitalise on conceptual metaphors related to children’s familiar contexts, rather than using more abstract representations. Such metaphorical connections, preferably building on physical properties, contribute to children’s comprehension and facilitate their exploration of the systems.
{"title":"Tangible Representational Properties: Implications for Meaning Making","authors":"Taciana Pontual Falcão","doi":"10.3390/mti2030054","DOIUrl":"https://doi.org/10.3390/mti2030054","url":null,"abstract":"Tangible technologies are considered promising tools for learning, by enabling multimodal interaction through physical action and manipulation of physical and digital elements, thus facilitating representational concrete–abstract links. A key concept in a tangible system is that its physical components are objects of interest, with associated meanings relevant to the context. Tangible technologies are said to provide ‘natural’ mappings that employ spatial analogies and adhere to cultural standards, capitalising on people’s familiarity with the physical world. Students with intellectual disabilities particularly benefit from interaction with tangibles, given their difficulties with perception and abstraction. However, symbolic information does not always have an obvious physical equivalent, and meanings do not reside in the representations used in the artefacts themselves, but in the ways they are manipulated and interpreted. In educational contexts, meaning attached to artefacts by designers is not necessarily transparent to students, nor interpreted by them as the designer predicted. Using artefacts and understanding their significance is of utmost importance for the construction of knowledge within the learning process; hence the need to study the use of the artefacts in contexts of practice and how they are transformed by the students. This article discusses how children with intellectual disabilities conceptually interpreted the elements of four tangible artefacts, and which characteristics of these tangibles were key for productive, multimodal interaction, thus potentially guiding designers and educators. Analysis shows the importance of designing physical-digital semantic mappings that capitalise on conceptual metaphors related to children’s familiar contexts, rather than using more abstract representations. Such metaphorical connections, preferably building on physical properties, contribute to children’s comprehension and facilitate their exploration of the systems.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/mti2030054","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45224953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：