首页 > 最新文献

Multimodal Technologies and Interaction最新文献

英文 中文
Reviews of Social Embodiment for Design of Non-Player Characters in Virtual Reality-Based Social Skill Training for Autistic Children 基于虚拟现实的自闭症儿童社交技能训练中非玩家角色设计的社会体现研究综述
IF 2.5 Q2 Computer Science Pub Date : 2018-09-04 DOI: 10.3390/MTI2030053
Jewoong Moon
The purpose of this paper is to review the scholarly works regarding social embodiment aligned with the design of non-player characters in virtual reality (VR)-based social skill training for autistic children. VR-based social skill training for autistic children has been a naturalistic environment, which allows autistic children themselves to shape socially-appropriate behaviors in real world. To build up the training environment for autistic children, it is necessary to identify how to simulate social components in the training. In particular, designing non-player characters (NPCs) in the training is essential to determining the quality of the simulated social interactions during the training. Through this literature review, this study proposes multiple design themes that underline the nature of social embodiment in which interactions with NPCs in VR-based social skill training take place.
本文旨在对基于虚拟现实(VR)的自闭症儿童社交技能训练中社会体现与非玩家角色设计相关的学术研究进行综述。基于虚拟现实的自闭症儿童社交技能训练是一种自然的环境,可以让自闭症儿童自己在现实世界中塑造适合社会的行为。要构建自闭症儿童的训练环境,必须明确如何在训练中模拟社会成分。特别是,在训练中设计非玩家角色(npc)对于决定训练过程中模拟社交互动的质量至关重要。通过这篇文献综述,本研究提出了多个设计主题,这些主题强调了在基于vr的社交技能训练中与npc互动的社会体现的本质。
{"title":"Reviews of Social Embodiment for Design of Non-Player Characters in Virtual Reality-Based Social Skill Training for Autistic Children","authors":"Jewoong Moon","doi":"10.3390/MTI2030053","DOIUrl":"https://doi.org/10.3390/MTI2030053","url":null,"abstract":"The purpose of this paper is to review the scholarly works regarding social embodiment aligned with the design of non-player characters in virtual reality (VR)-based social skill training for autistic children. VR-based social skill training for autistic children has been a naturalistic environment, which allows autistic children themselves to shape socially-appropriate behaviors in real world. To build up the training environment for autistic children, it is necessary to identify how to simulate social components in the training. In particular, designing non-player characters (NPCs) in the training is essential to determining the quality of the simulated social interactions during the training. Through this literature review, this study proposes multiple design themes that underline the nature of social embodiment in which interactions with NPCs in VR-based social skill training take place.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030053","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69756185","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Design for an Art Therapy Robot: An Explorative Review of the Theoretical Foundations for Engaging in Emotional and Creative Painting with a Robot 艺术治疗机器人的设计:探索用机器人进行情感和创造性绘画的理论基础
IF 2.5 Q2 Computer Science Pub Date : 2018-09-03 DOI: 10.3390/MTI2030052
M. Cooney, M. Menezes
Social robots are being designed to help support people’s well-being in domestic and public environments. To address increasing incidences of psychological and emotional difficulties such as loneliness, and a shortage of human healthcare workers, we believe that robots will also play a useful role in engaging with people in therapy, on an emotional and creative level, e.g., in music, drama, playing, and art therapy. Here, we focus on the latter case, on an autonomous robot capable of painting with a person. A challenge is that the theoretical foundations are highly complex; we are only just beginning ourselves to understand emotions and creativity in human science, which have been described as highly important challenges in artificial intelligence. To gain insight, we review some of the literature on robots used for therapy and art, potential strategies for interacting, and mechanisms for expressing emotions and creativity. In doing so, we also suggest the usefulness of the responsive art approach as a starting point for art therapy robots, describe a perceived gap between our understanding of emotions in human science and what is currently typically being addressed in engineering studies, and identify some potential ethical pitfalls and solutions for avoiding them. Based on our arguments, we propose a design for an art therapy robot, also discussing a simplified prototype implementation, toward informing future work in the area.
设计社交机器人是为了帮助人们在家庭和公共环境中获得幸福。为了解决越来越多的心理和情感问题,如孤独,以及人类医疗工作者的短缺,我们相信机器人也将在情感和创造性层面上与人们进行治疗,例如在音乐、戏剧、玩耍和艺术治疗中发挥有用的作用。在这里,我们关注的是后一种情况,即能够与人一起绘画的自主机器人。一个挑战是,理论基础非常复杂;我们才刚刚开始理解人类科学中的情感和创造力,这被描述为人工智能中非常重要的挑战。为了深入了解,我们回顾了一些关于机器人用于治疗和艺术的文献,潜在的互动策略,以及表达情感和创造力的机制。在此过程中,我们还建议将响应式艺术方法作为艺术治疗机器人的起点,描述我们对人类科学中情感的理解与目前工程研究中典型解决的问题之间的感知差距,并确定一些潜在的伦理陷阱和避免它们的解决方案。基于我们的论点,我们提出了一个艺术治疗机器人的设计,也讨论了一个简化的原型实现,为该领域的未来工作提供信息。
{"title":"Design for an Art Therapy Robot: An Explorative Review of the Theoretical Foundations for Engaging in Emotional and Creative Painting with a Robot","authors":"M. Cooney, M. Menezes","doi":"10.3390/MTI2030052","DOIUrl":"https://doi.org/10.3390/MTI2030052","url":null,"abstract":"Social robots are being designed to help support people’s well-being in domestic and public environments. To address increasing incidences of psychological and emotional difficulties such as loneliness, and a shortage of human healthcare workers, we believe that robots will also play a useful role in engaging with people in therapy, on an emotional and creative level, e.g., in music, drama, playing, and art therapy. Here, we focus on the latter case, on an autonomous robot capable of painting with a person. A challenge is that the theoretical foundations are highly complex; we are only just beginning ourselves to understand emotions and creativity in human science, which have been described as highly important challenges in artificial intelligence. To gain insight, we review some of the literature on robots used for therapy and art, potential strategies for interacting, and mechanisms for expressing emotions and creativity. In doing so, we also suggest the usefulness of the responsive art approach as a starting point for art therapy robots, describe a perceived gap between our understanding of emotions in human science and what is currently typically being addressed in engineering studies, and identify some potential ethical pitfalls and solutions for avoiding them. Based on our arguments, we propose a design for an art therapy robot, also discussing a simplified prototype implementation, toward informing future work in the area.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030052","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 19
Animals Make Music: A Look at Non-Human Musical Expression 动物创造音乐:非人类音乐表达的观察
IF 2.5 Q2 Computer Science Pub Date : 2018-09-02 DOI: 10.3390/MTI2030051
Reinhard Gupfinger, Martin Kaltenbrunner
The use of musical instruments and interfaces that involve animals in the interaction process is an emerging, yet not widespread practice. The projects that have been implemented in this unusual field are raising questions concerning ethical principles, animal-centered design processes, and the possible benefits and risks for the animals involved. Animal–Computer Interaction is a novel field of research that offers a framework (ACI manifesto) for implementing interactive technology for animals. Based on this framework, we have examined several projects focusing on the interplay between animals and music technology in order to arrive at a better understanding of animal-based musical projects. Building on this, we will discuss how the implementation of new musical instruments and interfaces could provide new opportunities for improving the quality of life for grey parrots living in captivity.
在互动过程中使用乐器和涉及动物的界面是一种新兴的做法,但并不普遍。在这个不同寻常的领域实施的项目引发了有关伦理原则、以动物为中心的设计过程以及所涉及动物可能的利益和风险的问题。动物-计算机交互是一个新颖的研究领域,它为实现动物交互技术提供了一个框架(ACI宣言)。基于这个框架,我们研究了几个专注于动物和音乐技术之间相互作用的项目,以便更好地理解基于动物的音乐项目。在此基础上,我们将讨论如何实施新的乐器和界面,为改善圈养灰鹦鹉的生活质量提供新的机会。
{"title":"Animals Make Music: A Look at Non-Human Musical Expression","authors":"Reinhard Gupfinger, Martin Kaltenbrunner","doi":"10.3390/MTI2030051","DOIUrl":"https://doi.org/10.3390/MTI2030051","url":null,"abstract":"The use of musical instruments and interfaces that involve animals in the interaction process is an emerging, yet not widespread practice. The projects that have been implemented in this unusual field are raising questions concerning ethical principles, animal-centered design processes, and the possible benefits and risks for the animals involved. Animal–Computer Interaction is a novel field of research that offers a framework (ACI manifesto) for implementing interactive technology for animals. Based on this framework, we have examined several projects focusing on the interplay between animals and music technology in order to arrive at a better understanding of animal-based musical projects. Building on this, we will discuss how the implementation of new musical instruments and interfaces could provide new opportunities for improving the quality of life for grey parrots living in captivity.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030051","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Deep Learning and Medical Diagnosis: A Review of Literature 深度学习与医学诊断:文献综述
IF 2.5 Q2 Computer Science Pub Date : 2018-08-17 DOI: 10.3390/MTI2030047
Mihalj Bakator, D. Radosav
In this review the application of deep learning for medical diagnosis is addressed. A thorough analysis of various scientific articles in the domain of deep neural networks application in the medical field has been conducted. More than 300 research articles were obtained, and after several selection steps, 46 articles were presented in more detail. The results indicate that convolutional neural networks (CNN) are the most widely represented when it comes to deep learning and medical image analysis. Furthermore, based on the findings of this article, it can be noted that the application of deep learning technology is widespread, but the majority of applications are focused on bioinformatics, medical diagnosis and other similar fields.
本文综述了深度学习在医学诊断中的应用。对深度神经网络在医学领域的应用进行了深入的分析。获得了300多篇研究论文,经过几个选择步骤,比较详细地介绍了46篇文章。结果表明,在深度学习和医学图像分析方面,卷积神经网络(CNN)是最广泛的代表。此外,根据本文的研究结果,可以注意到深度学习技术的应用是广泛的,但大多数应用集中在生物信息学,医学诊断等类似领域。
{"title":"Deep Learning and Medical Diagnosis: A Review of Literature","authors":"Mihalj Bakator, D. Radosav","doi":"10.3390/MTI2030047","DOIUrl":"https://doi.org/10.3390/MTI2030047","url":null,"abstract":"In this review the application of deep learning for medical diagnosis is addressed. A thorough analysis of various scientific articles in the domain of deep neural networks application in the medical field has been conducted. More than 300 research articles were obtained, and after several selection steps, 46 articles were presented in more detail. The results indicate that convolutional neural networks (CNN) are the most widely represented when it comes to deep learning and medical image analysis. Furthermore, based on the findings of this article, it can be noted that the application of deep learning technology is widespread, but the majority of applications are focused on bioinformatics, medical diagnosis and other similar fields.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030047","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 278
Technology for Remote Health Monitoring in an Older Population: A Role for Mobile Devices 老年人口远程健康监测技术:移动设备的作用
IF 2.5 Q2 Computer Science Pub Date : 2018-07-27 DOI: 10.3390/MTI2030043
Kate Dupuis, L. Tsotsos
The impact of an aging population on healthcare and the sustainability of our healthcare system are pressing issues in contemporary society. Technology has the potential to address these challenges, alleviating pressures on the healthcare system and empowering individuals to have greater control over monitoring their own health. Importantly, mobile devices such as smartphones and tablets can allow older adults to have “on the go” access to health-related information. This paper explores mobile health apps that enable older adults and those who care for them to track health-related factors such as body readings and medication adherence, and it serves as a review of the literature on the usability and acceptance of mobile health apps in an older population.
人口老龄化对医疗保健的影响以及我们医疗保健系统的可持续性是当代社会迫切需要解决的问题。技术有可能解决这些挑战,减轻医疗保健系统的压力,并使个人能够更好地控制自己的健康状况。重要的是,智能手机和平板电脑等移动设备可以让老年人“随时随地”获取与健康相关的信息。本文探讨了移动健康应用程序,这些应用程序使老年人和照顾他们的人能够跟踪与健康相关的因素,如身体读数和药物依从性,并且它可以作为对老年人移动健康应用程序的可用性和接受度的文献的回顾。
{"title":"Technology for Remote Health Monitoring in an Older Population: A Role for Mobile Devices","authors":"Kate Dupuis, L. Tsotsos","doi":"10.3390/MTI2030043","DOIUrl":"https://doi.org/10.3390/MTI2030043","url":null,"abstract":"The impact of an aging population on healthcare and the sustainability of our healthcare system are pressing issues in contemporary society. Technology has the potential to address these challenges, alleviating pressures on the healthcare system and empowering individuals to have greater control over monitoring their own health. Importantly, mobile devices such as smartphones and tablets can allow older adults to have “on the go” access to health-related information. This paper explores mobile health apps that enable older adults and those who care for them to track health-related factors such as body readings and medication adherence, and it serves as a review of the literature on the usability and acceptance of mobile health apps in an older population.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030043","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
Opportunities and Challenges of Bodily Interaction for Geometry Learning to Inform Technology Design 几何学习与技术设计的身体互动的机遇与挑战
IF 2.5 Q2 Computer Science Pub Date : 2018-07-09 DOI: 10.3390/MTI2030041
S. Price, S. Duffy
An increasing body of work provides evidence of the importance of bodily experience for cognition and the learning of mathematics. Sensor-based technologies have potential for guiding sensori-motor engagement with challenging mathematical ideas in new ways. Yet, designing environments that promote an appropriate sensori-motoric interaction that effectively supports salient foundations of mathematical concepts is challenging and requires understanding of opportunities and challenges that bodily interaction offers. This study aimed to better understand how young children can, and do, use their bodies to explore geometrical concepts of angle and shape, and what contribution the different sensori-motor experiences make to the comprehension of mathematical ideas. Twenty-nine students aged 6–10 years participated in an exploratory study, with paired and group activities designed to elicit intuitive bodily enactment of angles and shape. Our analysis, focusing on moment-by-moment bodily interactions, attended to gesture, action, facial expression, body posture and talk, illustrated the ‘realms of possibilities’ of bodily interaction, and highlighted challenges around ‘felt’ experience and egocentric vs. allocentric perception of the body during collaborative bodily enactment. These findings inform digital designs for sensory interaction to foreground salient geometric features and effectively support relevant forms of enactment to enhance the learning experience, supporting challenging aspects of interaction and exploiting the opportunities of the body.
越来越多的研究证明了身体体验对认知和数学学习的重要性。基于传感器的技术有可能以新的方式指导具有挑战性的数学思想的感觉-运动参与。然而,设计环境促进适当的感觉-运动相互作用,有效地支持数学概念的显著基础是具有挑战性的,需要理解身体相互作用提供的机会和挑战。本研究旨在更好地了解幼儿如何使用他们的身体来探索角度和形状的几何概念,以及不同的感觉运动体验对理解数学概念的贡献。29名6-10岁的学生参加了一项探索性研究,进行了配对和小组活动,旨在激发人们对角度和形状的直观身体行为。我们的分析集中于每时每刻的身体互动,包括手势、动作、面部表情、身体姿势和谈话,说明了身体互动的“可能性领域”,并强调了在协作身体表演过程中围绕“感觉”体验和自我中心与非中心的身体感知的挑战。这些发现为感官互动的数字设计提供了信息,以突出显著的几何特征,并有效地支持相关形式的制定,以增强学习体验,支持互动的挑战性方面,并利用身体的机会。
{"title":"Opportunities and Challenges of Bodily Interaction for Geometry Learning to Inform Technology Design","authors":"S. Price, S. Duffy","doi":"10.3390/MTI2030041","DOIUrl":"https://doi.org/10.3390/MTI2030041","url":null,"abstract":"An increasing body of work provides evidence of the importance of bodily experience for cognition and the learning of mathematics. Sensor-based technologies have potential for guiding sensori-motor engagement with challenging mathematical ideas in new ways. Yet, designing environments that promote an appropriate sensori-motoric interaction that effectively supports salient foundations of mathematical concepts is challenging and requires understanding of opportunities and challenges that bodily interaction offers. This study aimed to better understand how young children can, and do, use their bodies to explore geometrical concepts of angle and shape, and what contribution the different sensori-motor experiences make to the comprehension of mathematical ideas. Twenty-nine students aged 6–10 years participated in an exploratory study, with paired and group activities designed to elicit intuitive bodily enactment of angles and shape. Our analysis, focusing on moment-by-moment bodily interactions, attended to gesture, action, facial expression, body posture and talk, illustrated the ‘realms of possibilities’ of bodily interaction, and highlighted challenges around ‘felt’ experience and egocentric vs. allocentric perception of the body during collaborative bodily enactment. These findings inform digital designs for sensory interaction to foreground salient geometric features and effectively support relevant forms of enactment to enhance the learning experience, supporting challenging aspects of interaction and exploiting the opportunities of the body.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-07-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030041","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Animal-to-Animal Data Sharing Mechanism for Wildlife Monitoring in Fukushima Exclusion Zone 福岛隔离区野生动物监测动物间数据共享机制
IF 2.5 Q2 Computer Science Pub Date : 2018-07-03 DOI: 10.3390/MTI2030040
H. Kobayashi, Keijiro Nakagawa, K. Makiyama, Yuta Sasaki, Hiromi Kudo, Baburam Niraula, K. Sezaki
We propose an animal-to-animal data sharing mechanism that employs wildlife-borne sensing devices to expand the size of monitoring areas in which electricity, information, and road infrastructures are either limited or nonexistent. With the proposed approach, monitoring information can be collected from remote areas in a safe and cost-effective manner. To substantially prolong the life of a sensor node, the proposed mechanism activates the communication capabilities only when there is a plurality of animals; otherwise, the sensor node remains in a sleep state. This study aimed to achieve three objectives. First, we intend to obtain knowledge based on the actual field operations within the Fukushima exclusion zone. Second, we attempt to realize an objective evaluation of the power supply and work base that is required to properly evaluate the proposed mechanism. Third, we intend to acquire data to support wildlife research, which is the objective of both our present (and future) research.
我们提出了一种动物与动物之间的数据共享机制,该机制利用野生动物携带的传感设备来扩大电力、信息和道路基础设施有限或不存在的监测区域的规模。采用所提出的方法,可以以安全和具有成本效益的方式从偏远地区收集监测信息。为了大幅度延长传感器节点的寿命,所提出的机制仅在存在多个动物时激活通信能力;否则,传感器节点处于休眠状态。本研究旨在实现三个目标。首先,我们打算根据福岛禁区内的实际实地行动获得知识。其次,我们试图实现对电源和工作基础的客观评估,这是正确评估所提出的机制所需的。第三,我们打算获取支持野生动物研究的数据,这是我们目前(和未来)研究的目标。
{"title":"Animal-to-Animal Data Sharing Mechanism for Wildlife Monitoring in Fukushima Exclusion Zone","authors":"H. Kobayashi, Keijiro Nakagawa, K. Makiyama, Yuta Sasaki, Hiromi Kudo, Baburam Niraula, K. Sezaki","doi":"10.3390/MTI2030040","DOIUrl":"https://doi.org/10.3390/MTI2030040","url":null,"abstract":"We propose an animal-to-animal data sharing mechanism that employs wildlife-borne sensing devices to expand the size of monitoring areas in which electricity, information, and road infrastructures are either limited or nonexistent. With the proposed approach, monitoring information can be collected from remote areas in a safe and cost-effective manner. To substantially prolong the life of a sensor node, the proposed mechanism activates the communication capabilities only when there is a plurality of animals; otherwise, the sensor node remains in a sleep state. This study aimed to achieve three objectives. First, we intend to obtain knowledge based on the actual field operations within the Fukushima exclusion zone. Second, we attempt to realize an objective evaluation of the power supply and work base that is required to properly evaluate the proposed mechanism. Third, we intend to acquire data to support wildlife research, which is the objective of both our present (and future) research.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030040","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Exploring Emergent Features of Student Interaction within an Embodied Science Learning Simulation 探索具身科学学习模拟中学生互动的涌现特征
IF 2.5 Q2 Computer Science Pub Date : 2018-07-02 DOI: 10.3390/MTI2030039
Jina Kang, Robb Lindgren, James Planey
Theories of embodied cognition argue that human processes of thinking and reasoning are deeply connected with the actions and perceptions of the body. Recent research suggests that these theories can be successfully applied to the design of learning environments, and new technologies enable multimodal platforms that respond to students’ natural physical activity such as their gestures. This study examines how students engaged with an embodied mixed-reality science learning simulation using advanced gesture recognition techniques to support full-body interaction. The simulation environment acts as a communication platform for students to articulate their understanding of non-linear growth within different science contexts. In particular, this study investigates the different multimodal interaction metrics that were generated as students attempted to make sense of cross-cutting science concepts through using a personalized gesture scheme. Starting with video recordings of students’ full-body gestures, we examined the relationship between these embodied expressions and their subsequent success reasoning about non-linear growth. We report the patterns that we identified, and explicate our findings by detailing a few insightful cases of student interactions. Implications for the design of multimodal interaction technologies and the metrics that were used to investigate different types of students’ interactions while learning are discussed.
具身认知理论认为,人类的思考和推理过程与身体的行为和感知密切相关。最近的研究表明,这些理论可以成功地应用于学习环境的设计,新技术使多模式平台能够响应学生的自然身体活动,如他们的手势。本研究考察了学生如何使用先进的手势识别技术来支持全身互动,并参与到具体的混合现实科学学习模拟中。模拟环境作为一个交流平台,让学生在不同的科学背景下表达他们对非线性增长的理解。特别是,本研究调查了不同的多模态交互指标,这些指标是在学生试图通过使用个性化的手势方案来理解交叉科学概念时产生的。从学生全身手势的录像开始,我们研究了这些具身表达与他们随后对非线性增长的成功推理之间的关系。我们报告了我们发现的模式,并通过详细介绍几个有见地的学生互动案例来解释我们的发现。本文讨论了设计多模态交互技术的意义,以及用于调查不同类型学生在学习过程中的交互的指标。
{"title":"Exploring Emergent Features of Student Interaction within an Embodied Science Learning Simulation","authors":"Jina Kang, Robb Lindgren, James Planey","doi":"10.3390/MTI2030039","DOIUrl":"https://doi.org/10.3390/MTI2030039","url":null,"abstract":"Theories of embodied cognition argue that human processes of thinking and reasoning are deeply connected with the actions and perceptions of the body. Recent research suggests that these theories can be successfully applied to the design of learning environments, and new technologies enable multimodal platforms that respond to students’ natural physical activity such as their gestures. This study examines how students engaged with an embodied mixed-reality science learning simulation using advanced gesture recognition techniques to support full-body interaction. The simulation environment acts as a communication platform for students to articulate their understanding of non-linear growth within different science contexts. In particular, this study investigates the different multimodal interaction metrics that were generated as students attempted to make sense of cross-cutting science concepts through using a personalized gesture scheme. Starting with video recordings of students’ full-body gestures, we examined the relationship between these embodied expressions and their subsequent success reasoning about non-linear growth. We report the patterns that we identified, and explicate our findings by detailing a few insightful cases of student interactions. Implications for the design of multimodal interaction technologies and the metrics that were used to investigate different types of students’ interactions while learning are discussed.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030039","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
A Predictive Fingerstroke-Level Model for Smartwatch Interaction 智能手表交互的预测手指触控级模型
IF 2.5 Q2 Computer Science Pub Date : 2018-07-02 DOI: 10.3390/MTI2030038
Shiroq Al-Megren
The keystroke-level model (KLM) is commonly used to predict the time it will take an expert user to accomplish a task without errors when using an interactive system. The KLM was initially intended to predict interactions in conventional set-ups, i.e., mouse and keyboard interactions. However, it has since been adapted to predict interactions with smartphones, in-vehicle information systems, and natural user interfaces. The simplicity of the KLM and its extensions, along with their resource- and time-saving capabilities, has driven their adoption. In recent years, the popularity of smartwatches has grown, introducing new design challenges due to the small touch screens and bimanual interactions involved, which make current extensions to the KLM unsuitable for modelling smartwatches. Therefore, it is necessary to study these interfaces and interactions. This paper reports on three studies performed to modify the original KLM and its extensions for smartwatch interaction. First, an observational study was conducted to characterise smartwatch interactions. Second, the unit times for the observed interactions were derived through another study, in which the times required to perform the relevant physical actions were measured. Finally, a third study was carried out to validate the model for interactions with the Apple Watch and Samsung Gear S3. The results show that the new model can accurately predict the performance of smartwatch users with a percentage error of 12.07%; a value that falls below the acceptable percentage dictated by the original KLM ~21%.
在使用交互式系统时,通常使用击键级别模型(KLM)来预测专家用户在没有错误的情况下完成任务所需的时间。KLM最初旨在预测传统设置中的交互,即鼠标和键盘的交互。然而,它已经被用于预测与智能手机、车载信息系统和自然用户界面的交互。KLM及其扩展的简单性,以及它们节省资源和时间的功能,推动了它们的采用。近年来,智能手表越来越受欢迎,由于小型触摸屏和涉及的手动交互,引入了新的设计挑战,这使得目前对KLM的扩展不适合建模智能手表。因此,有必要对这些接口和交互进行研究。本文报告了三项研究,以修改原始KLM及其扩展的智能手表交互。首先,进行了一项观察性研究,以表征智能手表的互动。其次,观察到的相互作用的单位时间是通过另一项研究得出的,其中测量了执行相关物理动作所需的时间。最后,进行了第三项研究,以验证该模型与Apple Watch和Samsung Gear S3的交互。结果表明,新模型能够准确预测智能手表用户的行为,误差百分比为12.07%;低于原KLM规定的可接受百分比~21%的值。
{"title":"A Predictive Fingerstroke-Level Model for Smartwatch Interaction","authors":"Shiroq Al-Megren","doi":"10.3390/MTI2030038","DOIUrl":"https://doi.org/10.3390/MTI2030038","url":null,"abstract":"The keystroke-level model (KLM) is commonly used to predict the time it will take an expert user to accomplish a task without errors when using an interactive system. The KLM was initially intended to predict interactions in conventional set-ups, i.e., mouse and keyboard interactions. However, it has since been adapted to predict interactions with smartphones, in-vehicle information systems, and natural user interfaces. The simplicity of the KLM and its extensions, along with their resource- and time-saving capabilities, has driven their adoption. In recent years, the popularity of smartwatches has grown, introducing new design challenges due to the small touch screens and bimanual interactions involved, which make current extensions to the KLM unsuitable for modelling smartwatches. Therefore, it is necessary to study these interfaces and interactions. This paper reports on three studies performed to modify the original KLM and its extensions for smartwatch interaction. First, an observational study was conducted to characterise smartwatch interactions. Second, the unit times for the observed interactions were derived through another study, in which the times required to perform the relevant physical actions were measured. Finally, a third study was carried out to validate the model for interactions with the Apple Watch and Samsung Gear S3. The results show that the new model can accurately predict the performance of smartwatch users with a percentage error of 12.07%; a value that falls below the acceptable percentage dictated by the original KLM ~21%.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.3390/MTI2030038","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69755578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
Documenting the Elusive and Ephemeral in Embodied Design Ideation Activities 在具身设计创意活动中记录难以捉摸的和短暂的
IF 2.5 Q2 Computer Science Pub Date : 2018-06-24 DOI: 10.3390/mti2030035
Laia Turmo Vidal, Elena Márquez Segura
{"title":"Documenting the Elusive and Ephemeral in Embodied Design Ideation Activities","authors":"Laia Turmo Vidal, Elena Márquez Segura","doi":"10.3390/mti2030035","DOIUrl":"https://doi.org/10.3390/mti2030035","url":null,"abstract":"","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2018-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85032392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
期刊
Multimodal Technologies and Interaction
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1