首页 > 最新文献

Multimodal Technologies and Interaction最新文献

英文 中文
Optical Rules to Mitigate the Parallax-Related Registration Error in See-Through Head-Mounted Displays for the Guidance of Manual Tasks 在用于指导手动任务的透视头戴式显示器中减少与视差相关的注册误差的光学规则
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-01-04 DOI: 10.3390/mti8010004
Vincenzo Ferrari, N. Cattari, S. Condino, F. Cutolo
Head-mounted displays (HMDs) are hands-free devices particularly useful for guiding near-field tasks such as manual surgical procedures. See-through HMDs do not significantly alter the user’s direct view of the world, but the optical merging of real and virtual information can hinder their coherent and simultaneous perception. In particular, the coherence between the real and virtual content is affected by a viewpoint parallax-related misalignment, which is due to the inaccessibility of the user-perceived reality through the semi-transparent optical combiner of the OST Optical See-Through (OST) display. Recent works demonstrated that a proper selection of the collimation optics of the HMD significantly mitigates the parallax-related registration error without the need for any eye-tracking cameras and/or for any error-prone alignment-based display calibration procedures. These solutions are either based on HMDs that projects the virtual imaging plane directly at arm’s distance, or they require the integration on the HMD of additional lenses to optically move the image of the observed scene to the virtual projection plane of the HMD. This paper describes and evaluates the pros and cons of both the suggested solutions by providing an analytical estimation of the residual registration error achieved with both solutions and discussing the perceptual issues generated by the simultaneous focalization of real and virtual information.
头戴式显示器(HMD)是一种免提设备,特别适用于指导近场任务,如手工外科手术。透视头戴式显示器不会明显改变用户对世界的直接观察,但真实信息和虚拟信息的光学融合会妨碍它们的连贯和同步感知。特别是,真实和虚拟内容之间的一致性会受到视点视差相关错位的影响,这是由于用户无法通过 OST 光学透视(OST)显示器的半透明光学组合器获得所感知的现实。最近的研究表明,适当选择 HMD 的准直光学器件可显著减少视差相关的配准误差,而无需任何眼球跟踪摄像头和/或任何易出错的基于准直的显示校准程序。这些解决方案要么基于直接在手臂距离处投射虚拟成像平面的 HMD,要么需要在 HMD 上集成额外的透镜,以光学方式将观察到的场景图像移动到 HMD 的虚拟投影平面。本文描述并评估了这两种建议解决方案的利弊,对两种解决方案所实现的残余配准误差进行了分析估计,并讨论了真实和虚拟信息同时聚焦所产生的感知问题。
{"title":"Optical Rules to Mitigate the Parallax-Related Registration Error in See-Through Head-Mounted Displays for the Guidance of Manual Tasks","authors":"Vincenzo Ferrari, N. Cattari, S. Condino, F. Cutolo","doi":"10.3390/mti8010004","DOIUrl":"https://doi.org/10.3390/mti8010004","url":null,"abstract":"Head-mounted displays (HMDs) are hands-free devices particularly useful for guiding near-field tasks such as manual surgical procedures. See-through HMDs do not significantly alter the user’s direct view of the world, but the optical merging of real and virtual information can hinder their coherent and simultaneous perception. In particular, the coherence between the real and virtual content is affected by a viewpoint parallax-related misalignment, which is due to the inaccessibility of the user-perceived reality through the semi-transparent optical combiner of the OST Optical See-Through (OST) display. Recent works demonstrated that a proper selection of the collimation optics of the HMD significantly mitigates the parallax-related registration error without the need for any eye-tracking cameras and/or for any error-prone alignment-based display calibration procedures. These solutions are either based on HMDs that projects the virtual imaging plane directly at arm’s distance, or they require the integration on the HMD of additional lenses to optically move the image of the observed scene to the virtual projection plane of the HMD. This paper describes and evaluates the pros and cons of both the suggested solutions by providing an analytical estimation of the residual registration error achieved with both solutions and discussing the perceptual issues generated by the simultaneous focalization of real and virtual information.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"67 23","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-01-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139385512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Virtual Reality Assessment of Attention Deficits in Traumatic Brain Injury: Effectiveness and Ecological Validity 创伤性脑损伤患者注意力缺陷的虚拟现实评估:有效性和生态有效性
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2024-01-03 DOI: 10.3390/mti8010003
A. Malegiannaki, Evangelia Garefalaki, Nikolaos Pellas, M. Kosmidis
Early detection is crucial for addressing attention deficits commonly associated with Traumatic brain injury (TBI), informing effective rehabilitation planning and intervention. While traditional neuropsychological assessments have been conventionally used to evaluate attention deficits, their limited ecological validity presents notable challenges. This study explores the efficacy and validity of a novel virtual reality test, the Computerized Battery for the Assessment of Attention Disorders (CBAAD), among a cohort of TBI survivors (n = 20), in comparison to a healthy control group (n = 20). Participants, ranging in age from 21 to 62 years, were administered a comprehensive neuropsychological assessment, including the CBAAD and the Attention Related Cognitive Errors Scale. While variations in attentional performance were observed across age cohorts, the study found no statistically significant age-related effects within either group. The CBAAD demonstrated sensitivity to attentional dysfunction in the TBI group, establishing its value as a comprehensive test battery for assessing attention in this specific population. Regression analyses demonstrated the CBAAD’s effectiveness in predicting real-life attentional errors reported by TBI patients. In summary, the CBAAD demonstrates sensitivity to attentional dysfunction in TBI patients and the ability to predict real-world attentional errors, establishing its value as a comprehensive test battery for assessing attention in this specific population. Its implementation holds promise for enhancing the early identification of attentional impairments and facilitating tailored rehabilitation strategies for TBI patients.
早期发现对于解决与创伤性脑损伤(TBI)常见的注意力缺陷至关重要,可为有效的康复规划和干预提供依据。虽然传统的神经心理学评估一直被用于评估注意力缺陷,但其有限的生态有效性带来了显著的挑战。本研究探讨了一种新型虚拟现实测试--注意力障碍评估计算机化电池(CBAAD)--与健康对照组(20 人)相比,在一组 TBI 幸存者(20 人)中的功效和有效性。参与者的年龄从 21 岁到 62 岁不等,他们接受了全面的神经心理学评估,包括注意力障碍评估测试工具和注意力相关认知错误量表。虽然不同年龄组的注意力表现存在差异,但研究发现这两个组别在统计学上都没有明显的年龄相关效应。注意力相关认知错误量表(CBAAD)对创伤性脑损伤组的注意力功能障碍具有敏感性,从而确立了其作为评估这一特殊人群注意力的综合测试组合的价值。回归分析表明,CBAAD 能有效预测 TBI 患者报告的现实生活中的注意力错误。总之,CBAAD 显示了对 TBI 患者注意力功能障碍的敏感性和预测真实世界注意力错误的能力,从而确立了其作为评估这类特殊人群注意力的综合测试组合的价值。它的实施有望加强对注意力障碍的早期识别,并促进为创伤性脑损伤患者量身定制康复策略。
{"title":"Virtual Reality Assessment of Attention Deficits in Traumatic Brain Injury: Effectiveness and Ecological Validity","authors":"A. Malegiannaki, Evangelia Garefalaki, Nikolaos Pellas, M. Kosmidis","doi":"10.3390/mti8010003","DOIUrl":"https://doi.org/10.3390/mti8010003","url":null,"abstract":"Early detection is crucial for addressing attention deficits commonly associated with Traumatic brain injury (TBI), informing effective rehabilitation planning and intervention. While traditional neuropsychological assessments have been conventionally used to evaluate attention deficits, their limited ecological validity presents notable challenges. This study explores the efficacy and validity of a novel virtual reality test, the Computerized Battery for the Assessment of Attention Disorders (CBAAD), among a cohort of TBI survivors (n = 20), in comparison to a healthy control group (n = 20). Participants, ranging in age from 21 to 62 years, were administered a comprehensive neuropsychological assessment, including the CBAAD and the Attention Related Cognitive Errors Scale. While variations in attentional performance were observed across age cohorts, the study found no statistically significant age-related effects within either group. The CBAAD demonstrated sensitivity to attentional dysfunction in the TBI group, establishing its value as a comprehensive test battery for assessing attention in this specific population. Regression analyses demonstrated the CBAAD’s effectiveness in predicting real-life attentional errors reported by TBI patients. In summary, the CBAAD demonstrates sensitivity to attentional dysfunction in TBI patients and the ability to predict real-world attentional errors, establishing its value as a comprehensive test battery for assessing attention in this specific population. Its implementation holds promise for enhancing the early identification of attentional impairments and facilitating tailored rehabilitation strategies for TBI patients.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"14 7","pages":""},"PeriodicalIF":2.5,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139389196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Applying Cognitive Load Theory to eLearning of Crafts 将认知负荷理论应用于工艺电子学习
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-29 DOI: 10.3390/mti8010002
Nikolaos Partarakis, X. Zabulis
Craft education and training are important for preserving cultural heritage and fostering artisanal skills. However, the pedagogical challenges in this domain are numerous. This research paper presents a comprehensive framework for applying Cognitive Load Theory to enhance craft education and training via eLearning platforms. In this study, practical guidelines based on CLT principles are provided to optimize the instructional design and content delivery. These guidelines scaffold craft learning experiences within eLearning platforms and encompass strategies to manage cognitive load, promote active learning, and facilitate gradual transition. Subsequently, the paper details the implementation of these guidelines within a popular eLearning platform, Moodle, emphasizing its adaptability and utility for craft education. It discusses the customization of Moodle courses to align with the cognitive load management principles, providing a practical blueprint for educators and instructional designers. The research culminates in a case study, wherein the guidelines are applied to a craft eLearning course using Moodle.
手工艺教育和培训对于保护文化遗产和培养手工艺技能非常重要。然而,这一领域在教学方面存在诸多挑战。本研究论文提出了一个应用认知负荷理论的综合框架,以通过电子学习平台加强手工艺教育和培训。本研究提供了基于认知负荷理论原则的实用指南,以优化教学设计和内容交付。这些指导原则为电子学习平台中的工艺学习体验提供了支架,并包含了管理认知负荷、促进主动学习和推动渐进过渡的策略。随后,论文详细介绍了在流行的电子学习平台 Moodle 中实施这些指导原则的情况,强调了其对工艺教育的适应性和实用性。论文讨论了如何根据认知负荷管理原则定制 Moodle 课程,为教育工作者和教学设计师提供了一个实用的蓝图。研究最后还进行了案例分析,将指导原则应用于使用 Moodle 的手工艺电子学习课程。
{"title":"Applying Cognitive Load Theory to eLearning of Crafts","authors":"Nikolaos Partarakis, X. Zabulis","doi":"10.3390/mti8010002","DOIUrl":"https://doi.org/10.3390/mti8010002","url":null,"abstract":"Craft education and training are important for preserving cultural heritage and fostering artisanal skills. However, the pedagogical challenges in this domain are numerous. This research paper presents a comprehensive framework for applying Cognitive Load Theory to enhance craft education and training via eLearning platforms. In this study, practical guidelines based on CLT principles are provided to optimize the instructional design and content delivery. These guidelines scaffold craft learning experiences within eLearning platforms and encompass strategies to manage cognitive load, promote active learning, and facilitate gradual transition. Subsequently, the paper details the implementation of these guidelines within a popular eLearning platform, Moodle, emphasizing its adaptability and utility for craft education. It discusses the customization of Moodle courses to align with the cognitive load management principles, providing a practical blueprint for educators and instructional designers. The research culminates in a case study, wherein the guidelines are applied to a craft eLearning course using Moodle.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":" 32","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139142533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Role of Haptics in Training and Games for Hearing-Impaired Individuals: A Systematic Review 触觉技术在听障人士培训和游戏中的作用:系统回顾
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-22 DOI: 10.3390/mti8010001
Francesco Ganis, Andrea Gulli, Federico Fontana, Stefania Serafin
Sensory substitution and augmentation are pivotal concepts in multi-modal perception, particularly when confronting the challenges associated with impaired or missing sense rehabilitation. The present systematic review investigates the role of haptics for the hearing impaired in training or gamified activities. We applied a set of keywords to the Scopus® and PubMed® databases, obtaining a collection of 35 manuscripts spanning 23 years. Each article has been categorized following a documented procedure and thoroughly analyzed. Our findings reveal a rising number of studies in this field in the last five years, mostly testing the effectiveness of the developed rehabilitative method (77.14%). Despite a wide variety in almost every category we analyzed, such as haptic devices, body location, and data collection, we report a constant difficulty in recruitment, reflected in the low number of hearing-impaired participants (mean of 8.31). This review found that in all six papers reporting statistically significant positive results, the vibrotactile device in use generated vibrations starting from a sound, suggesting that some perceptual aspects connected to sound are transmittable through touch. This fact provides evidence that haptics and vibrotactile devices could be viable solutions for hearing-impaired rehabilitation and training.
感官替代和增强是多模态感知的关键概念,尤其是在面对与感官受损或缺失相关的康复挑战时。本系统综述调查了触觉技术在听力受损者的训练或游戏化活动中的作用。我们在 Scopus® 和 PubMed® 数据库中应用了一组关键词,收集到 35 篇手稿,时间跨度长达 23 年。每篇文章都按照记录程序进行了分类,并进行了全面分析。我们的研究结果表明,在过去的五年中,该领域的研究数量不断增加,其中大部分都是测试所开发的康复方法的有效性(77.14%)。尽管我们分析了几乎所有类别的研究,如触觉设备、身体位置和数据收集等,但我们发现在招募参与者方面一直存在困难,这反映在听障参与者的数量较少(平均为 8.31 人)。本综述发现,在所有六篇报告了具有统计学意义的积极结果的论文中,所使用的振动触觉设备都是从声音开始产生振动的,这表明与声音相关的某些感知方面是可以通过触觉传递的。这一事实证明,触觉和振动触觉设备可以成为听障康复和训练的可行解决方案。
{"title":"The Role of Haptics in Training and Games for Hearing-Impaired Individuals: A Systematic Review","authors":"Francesco Ganis, Andrea Gulli, Federico Fontana, Stefania Serafin","doi":"10.3390/mti8010001","DOIUrl":"https://doi.org/10.3390/mti8010001","url":null,"abstract":"Sensory substitution and augmentation are pivotal concepts in multi-modal perception, particularly when confronting the challenges associated with impaired or missing sense rehabilitation. The present systematic review investigates the role of haptics for the hearing impaired in training or gamified activities. We applied a set of keywords to the Scopus® and PubMed® databases, obtaining a collection of 35 manuscripts spanning 23 years. Each article has been categorized following a documented procedure and thoroughly analyzed. Our findings reveal a rising number of studies in this field in the last five years, mostly testing the effectiveness of the developed rehabilitative method (77.14%). Despite a wide variety in almost every category we analyzed, such as haptic devices, body location, and data collection, we report a constant difficulty in recruitment, reflected in the low number of hearing-impaired participants (mean of 8.31). This review found that in all six papers reporting statistically significant positive results, the vibrotactile device in use generated vibrations starting from a sound, suggesting that some perceptual aspects connected to sound are transmittable through touch. This fact provides evidence that haptics and vibrotactile devices could be viable solutions for hearing-impaired rehabilitation and training.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"21 8","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138947158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
“A Safe Space for Sharing Feelings”: Perspectives of Children with Lived Experiences of Anxiety on Social Robots "分享感受的安全空间":有焦虑生活经历的儿童对社交机器人的看法
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-15 DOI: 10.3390/mti7120118
Jill A. Dosso, Jaya N. Kailley, Susanna E. Martin, Julie M. Robillard
Social robots have the potential to support health and quality of life for children experiencing anxiety. We engaged families with lived experiences of pediatric anxiety in social robot development to explore desired design features, application areas, and emotion functionalities of social robots in anxiety care. We conducted 10 online co-creation workshops with (1) children with anxiety aged 7–13 (n = 24) with their family members (n = 20), and (2) youth with anxiety aged 14–18 (n = 12). Workshop participation included a validated robot expectations scale, anonymous polls, and discussion. Transcripts and text responses were subjected to content analysis. A lived experience expert group provided feedback throughout the research. Participants desired a pet-like robot with a soft texture, expressive eyes, and emotion detection to support activities of daily living. Specific anxiety-related applications included breathing exercises, managing distressing thoughts, and encouragement. Emotional alignment, the design of a robot’s emotional display, and the emotional impacts of an interaction were discussed. Privacy and the replacement of human interaction were concerns. We identify pediatric anxiety-specific design features, applications, and affective considerations for existing and future social robots. Our findings highlight the need for customizability and robust emotional functionality in social robot technologies intended to support the health and care of children living with anxiety.
社交机器人有可能为焦虑儿童的健康和生活质量提供支持。我们让有儿科焦虑症生活经历的家庭参与到社交机器人的开发中,探索社交机器人在焦虑症护理中的理想设计特点、应用领域和情感功能。我们与(1)7-13 岁的焦虑症儿童(24 人)及其家庭成员(20 人)和(2)14-18 岁的焦虑症青少年(12 人)开展了 10 次在线共同创造研讨会。参加研讨会的内容包括经过验证的机器人期望量表、匿名投票和讨论。对笔录和文本回复进行了内容分析。生活经验专家组在整个研究过程中提供反馈。参与者希望机器人能像宠物一样,具有柔软的质地、富有表现力的眼睛和情绪检测功能,以支持日常生活活动。与焦虑相关的具体应用包括呼吸练习、管理苦恼的想法和鼓励。会上还讨论了情绪调整、机器人情绪显示的设计以及互动对情绪的影响。隐私和人机交互的替代问题也受到了关注。我们为现有和未来的社交机器人确定了针对儿科焦虑症的设计特点、应用和情感考虑因素。我们的研究结果突出表明,社交机器人技术需要具备可定制性和强大的情感功能,以支持焦虑症儿童的健康和护理。
{"title":"“A Safe Space for Sharing Feelings”: Perspectives of Children with Lived Experiences of Anxiety on Social Robots","authors":"Jill A. Dosso, Jaya N. Kailley, Susanna E. Martin, Julie M. Robillard","doi":"10.3390/mti7120118","DOIUrl":"https://doi.org/10.3390/mti7120118","url":null,"abstract":"Social robots have the potential to support health and quality of life for children experiencing anxiety. We engaged families with lived experiences of pediatric anxiety in social robot development to explore desired design features, application areas, and emotion functionalities of social robots in anxiety care. We conducted 10 online co-creation workshops with (1) children with anxiety aged 7–13 (n = 24) with their family members (n = 20), and (2) youth with anxiety aged 14–18 (n = 12). Workshop participation included a validated robot expectations scale, anonymous polls, and discussion. Transcripts and text responses were subjected to content analysis. A lived experience expert group provided feedback throughout the research. Participants desired a pet-like robot with a soft texture, expressive eyes, and emotion detection to support activities of daily living. Specific anxiety-related applications included breathing exercises, managing distressing thoughts, and encouragement. Emotional alignment, the design of a robot’s emotional display, and the emotional impacts of an interaction were discussed. Privacy and the replacement of human interaction were concerns. We identify pediatric anxiety-specific design features, applications, and affective considerations for existing and future social robots. Our findings highlight the need for customizability and robust emotional functionality in social robot technologies intended to support the health and care of children living with anxiety.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"294 4","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138996841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Authoring Moving Parts of Objects in AR, VR and the Desktop 在 AR、VR 和桌面中制作物体的移动部件
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-13 DOI: 10.3390/mti7120117
Andrés N. Vargas González, Brian Williamson, J. Laviola
Creating digital twins of real objects is becoming more popular, with smartphones providing 3D scanning capabilities. Adding semantics to the reconstructed virtual objects is important to possibly reproduce training scenarios that otherwise could demand significant resources or happen in dangerous scenarios in some cases. The aim of this work is to evaluate the usability of authoring object component behaviors in immersive and non-immersive approaches. Therefore, we present an evaluation of the perceived ease of use to author moving parts of objects under three different conditions: desktop, augmented reality (AR) and virtual reality (VR). This paper provides insights into the perceived benefits and issues that domain experts might encounter when authoring geometrical component behaviors across each interface. A within-subject study is the major contribution of this work, from which is presented an analysis based on the usability, workload and user interface preferences of participants in the study. To reduce confounding variables in the study, we ensured that the virtual objects and the environment used for the evaluation were digital twins of the real objects and the environment that the experiment took place in. Results show that the desktop interface was perceived as more efficient and easier to use based on usability and workload measures. The desktop was preferred for performing component selection but no difference was found in the preference for defining a behavior and visualizing it. Based on these results, a set of recommendations and future directions are provided to achieve a more usable, immersive authoring experience.
借助智能手机提供的 3D 扫描功能,创建真实物体的数字双胞胎正变得越来越流行。为重建的虚拟物体添加语义对于重现训练场景非常重要,否则可能需要大量资源或在某些情况下发生危险。这项工作的目的是评估在沉浸式和非沉浸式方法中编写对象组件行为的可用性。因此,我们对在桌面、增强现实(AR)和虚拟现实(VR)三种不同条件下创作物体运动部件的易用性进行了评估。本文深入探讨了领域专家在每个界面上创作几何组件行为时可能遇到的感知优势和问题。这项工作的主要贡献在于开展了一项主体内研究,并根据研究参与者的可用性、工作量和用户界面偏好进行了分析。为了减少研究中的混杂变量,我们确保用于评估的虚拟对象和环境是实验中真实对象和环境的数字双胞胎。结果表明,根据可用性和工作量的衡量标准,人们认为桌面界面更高效、更易用。在进行组件选择时,人们更倾向于使用桌面界面,但在定义行为和可视化行为方面,人们的偏好没有差别。基于这些结果,我们提出了一系列建议和未来发展方向,以实现更可用、更身临其境的创作体验。
{"title":"Authoring Moving Parts of Objects in AR, VR and the Desktop","authors":"Andrés N. Vargas González, Brian Williamson, J. Laviola","doi":"10.3390/mti7120117","DOIUrl":"https://doi.org/10.3390/mti7120117","url":null,"abstract":"Creating digital twins of real objects is becoming more popular, with smartphones providing 3D scanning capabilities. Adding semantics to the reconstructed virtual objects is important to possibly reproduce training scenarios that otherwise could demand significant resources or happen in dangerous scenarios in some cases. The aim of this work is to evaluate the usability of authoring object component behaviors in immersive and non-immersive approaches. Therefore, we present an evaluation of the perceived ease of use to author moving parts of objects under three different conditions: desktop, augmented reality (AR) and virtual reality (VR). This paper provides insights into the perceived benefits and issues that domain experts might encounter when authoring geometrical component behaviors across each interface. A within-subject study is the major contribution of this work, from which is presented an analysis based on the usability, workload and user interface preferences of participants in the study. To reduce confounding variables in the study, we ensured that the virtual objects and the environment used for the evaluation were digital twins of the real objects and the environment that the experiment took place in. Results show that the desktop interface was perceived as more efficient and easier to use based on usability and workload measures. The desktop was preferred for performing component selection but no difference was found in the preference for defining a behavior and visualizing it. Based on these results, a set of recommendations and future directions are provided to achieve a more usable, immersive authoring experience.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"123 3","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139004441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Teachers’ Perspectives on Using Augmented-Reality-Enhanced Analytics as a Measure of Student Disengagement 教师对使用增强现实增强分析法衡量学生脱离课堂情况的看法
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-13 DOI: 10.3390/mti7120116
Manjeet Singh, Shaun Bangay, Atul Sajjanhar
There are various ways that teachers manage student disengagement levels during their class lessons, and managing disengagement can be both stressful and challenging, especially since each student is unique. Methods and techniques utilised are specific to teachers’ own experience level, subject knowledge, and teaching styles. We report on the techniques and methods teachers utilise to identify, mitigate, and measure student disengagement during class lessons; the paper presents the results of a mixed-methods, multisession study design comprising gathered qualitative and quantitative data to enable a greater understanding. Eight educators who were full-time educators with varying years of experience from three different schools, who taught or had taught English, maths, and science subjects at the primary school level, participated in this study. The study also observed that teachers used three AR applications and collected valuable feedback on their perspectives by using analytics generated by AR applications to help manage student disengagement. A postsession survey tool was used to gather the perceived importance and ranking of the techniques and methods discussed by the teachers during the previous sessions. The results showed that the majority of teachers deemed spending “Time on Tasks” and giving “Feedback/Reflections” most suited for measuring disengagement, and encouraging “Movement” and use of “Technology” emerged as the most favoured for mitigating disengagement. For utilising AR enhanced analytics in mitigating and measuring student disengagement, the data suggested a difference in perspectives based on teachers’ teaching levels, especially concerning conversations and the use of technology devices. The study did not find conclusive evidence of differences based on teachers’ teaching subjects and there was a notable distinction in building positive relationships among English teachers. This leads to the suggestion that subject-specific pedagogy might influence the perceived effectiveness of using AR-generated analytics in mitigating and measuring student disengagement.
教师在课堂上管理学生脱离课堂的程度有多种方法,管理学生脱离课堂既有压力又有挑战性,特别是因为每个学生都是独一无二的。所使用的方法和技巧与教师自身的经验水平、学科知识和教学风格有关。我们报告了教师在课堂教学中识别、缓解和衡量学生脱离课堂情况的技巧和方法;本文介绍了混合方法、多时段研究设计的结果,包括收集的定性和定量数据,以便加深理解。来自三所不同学校的八位具有不同工作年限的全职教育工作者参与了本研究,他们在小学阶段教授或曾经教授过英语、数学和科学科目。研究还观察到,教师使用了三种 AR 应用程序,并通过使用 AR 应用程序生成的分析数据收集了宝贵的反馈意见,以帮助管理学生的脱离情况。研究使用了课后调查工具,以收集教师对前几节课所讨论的技巧和方法的认知重要性和排名。结果显示,大多数教师认为 "花时间完成任务 "和 "反馈/反思 "最适合用来衡量脱离情况,而鼓励 "运动 "和使用 "技术 "则是最受欢迎的缓解脱离情况的方法。对于利用增强现实技术分析来缓解和衡量学生脱离课堂的情况,数据表明,教师的教学水平不同,其观点也不同,特别是在对话和使用技术设备方面。研究并未发现教师教学科目差异的确凿证据,但英语教师在建立积极关系方面存在明显差异。这就表明,特定学科的教学法可能会影响使用 AR 生成的分析在缓解和衡量学生脱离课堂情况方面的效果。
{"title":"Teachers’ Perspectives on Using Augmented-Reality-Enhanced Analytics as a Measure of Student Disengagement","authors":"Manjeet Singh, Shaun Bangay, Atul Sajjanhar","doi":"10.3390/mti7120116","DOIUrl":"https://doi.org/10.3390/mti7120116","url":null,"abstract":"There are various ways that teachers manage student disengagement levels during their class lessons, and managing disengagement can be both stressful and challenging, especially since each student is unique. Methods and techniques utilised are specific to teachers’ own experience level, subject knowledge, and teaching styles. We report on the techniques and methods teachers utilise to identify, mitigate, and measure student disengagement during class lessons; the paper presents the results of a mixed-methods, multisession study design comprising gathered qualitative and quantitative data to enable a greater understanding. Eight educators who were full-time educators with varying years of experience from three different schools, who taught or had taught English, maths, and science subjects at the primary school level, participated in this study. The study also observed that teachers used three AR applications and collected valuable feedback on their perspectives by using analytics generated by AR applications to help manage student disengagement. A postsession survey tool was used to gather the perceived importance and ranking of the techniques and methods discussed by the teachers during the previous sessions. The results showed that the majority of teachers deemed spending “Time on Tasks” and giving “Feedback/Reflections” most suited for measuring disengagement, and encouraging “Movement” and use of “Technology” emerged as the most favoured for mitigating disengagement. For utilising AR enhanced analytics in mitigating and measuring student disengagement, the data suggested a difference in perspectives based on teachers’ teaching levels, especially concerning conversations and the use of technology devices. The study did not find conclusive evidence of differences based on teachers’ teaching subjects and there was a notable distinction in building positive relationships among English teachers. This leads to the suggestion that subject-specific pedagogy might influence the perceived effectiveness of using AR-generated analytics in mitigating and measuring student disengagement.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"16 5","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139005470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sharing Cultural Heritage—The Case of the Lodovico Media Library 共享文化遗产--洛多维克媒体图书馆案例
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-05 DOI: 10.3390/mti7120115
Matteo Al Kalak, Lorenzo Baraldi
The article aims to reflect on the Lodovico media library, a digital repository preserving the digitised cultural heritage of the Emilia-Romagna region. The first part covers the project’s history and the challenges encountered during its setup phase, and we also explore the co-creation approach employed in defining the metadata architecture. The discussion extends by outlining the key features of shared metadata, illustrating their application to diverse digital objects within the Lodovico media library. Following a concise examination of the methodology for collecting/creating data and the initial research findings, the article concludes by highlighting the project’s potential in the realm of automatic handwriting recognition processes.
本文旨在反思Lodovico媒体图书馆,这是一个保存艾米利亚-罗马涅地区数字化文化遗产的数字存储库。第一部分介绍了项目的历史和在其设置阶段遇到的挑战,我们还探讨了在定义元数据体系结构时采用的共同创建方法。讨论通过概述共享元数据的关键特性进行扩展,说明它们在Lodovico媒体库中的各种数字对象上的应用。在对收集/创建数据的方法和初步研究结果进行了简明的检查之后,文章最后强调了该项目在自动手写识别过程领域的潜力。
{"title":"Sharing Cultural Heritage—The Case of the Lodovico Media Library","authors":"Matteo Al Kalak, Lorenzo Baraldi","doi":"10.3390/mti7120115","DOIUrl":"https://doi.org/10.3390/mti7120115","url":null,"abstract":"The article aims to reflect on the Lodovico media library, a digital repository preserving the digitised cultural heritage of the Emilia-Romagna region. The first part covers the project’s history and the challenges encountered during its setup phase, and we also explore the co-creation approach employed in defining the metadata architecture. The discussion extends by outlining the key features of shared metadata, illustrating their application to diverse digital objects within the Lodovico media library. Following a concise examination of the methodology for collecting/creating data and the initial research findings, the article concludes by highlighting the project’s potential in the realm of automatic handwriting recognition processes.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"3 3","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138598026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Enhancing Localization Performance with Extended Funneling Vibrotactile Feedback 利用扩展漏斗式振动反馈提高定位性能
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-05 DOI: 10.3390/mti7120114
Kalliopi Apostolou, Filip Škola, F. Liarokapis
This study explores the conventional ‘funneling’ method by introducing two extra locations beyond the virtual reality (VR) controller boundaries, terming it the extended funneling technique. Thirty-two participants engaged in a localization task, with their responses recorded using eye-tracking technology. They were tasked with localizing a virtual ping-pong ball as it bounced both within and outside their virtual hands on a virtual board. Both the experimental and control groups received simultaneous spatial audio and vibrotactile feedback. The experimental group received vibrotactile feedback with extended funneling, while the control group received vibrotactile feedback without funneling for comparison. The results indicate that the experimental group, benefiting from the extended funneling technique, demonstrated a significantly higher accuracy rate (41.79%) in localizing audio–vibrotactile stimuli compared to the control group (28.21%). No significant differences emerged in embodiment or workload scores. These findings highlight the effectiveness of extended funneling for enhancing the localization of sensory stimuli in VR.
本研究通过在虚拟现实(VR)控制器边界之外引入两个额外的位置来探索传统的“漏斗”方法,将其称为扩展漏斗技术。32名参与者参与了一项定位任务,他们的反应用眼球追踪技术记录下来。他们的任务是定位一个虚拟的乒乓球,当它在虚拟的棋盘上在他们的虚拟手上和外面弹跳时。实验组和对照组同时收到空间音频和触觉振动反馈。实验组采用带扩展漏斗的触觉振动反馈,对照组采用不带漏斗的触觉振动反馈进行对比。实验结果表明,利用扩展漏斗技术,实验组对听觉-振动-触觉刺激的定位准确率(41.79%)明显高于对照组(28.21%)。在体现或工作负荷得分上没有出现显著差异。这些发现强调了扩展漏斗在增强VR感官刺激定位方面的有效性。
{"title":"Enhancing Localization Performance with Extended Funneling Vibrotactile Feedback","authors":"Kalliopi Apostolou, Filip Škola, F. Liarokapis","doi":"10.3390/mti7120114","DOIUrl":"https://doi.org/10.3390/mti7120114","url":null,"abstract":"This study explores the conventional ‘funneling’ method by introducing two extra locations beyond the virtual reality (VR) controller boundaries, terming it the extended funneling technique. Thirty-two participants engaged in a localization task, with their responses recorded using eye-tracking technology. They were tasked with localizing a virtual ping-pong ball as it bounced both within and outside their virtual hands on a virtual board. Both the experimental and control groups received simultaneous spatial audio and vibrotactile feedback. The experimental group received vibrotactile feedback with extended funneling, while the control group received vibrotactile feedback without funneling for comparison. The results indicate that the experimental group, benefiting from the extended funneling technique, demonstrated a significantly higher accuracy rate (41.79%) in localizing audio–vibrotactile stimuli compared to the control group (28.21%). No significant differences emerged in embodiment or workload scores. These findings highlight the effectiveness of extended funneling for enhancing the localization of sensory stimuli in VR.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":"83 13","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138600324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Kids Save Lives by Learning through a Serious Game 儿童通过严肃游戏学习拯救生命
IF 2.5 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pub Date : 2023-12-01 DOI: 10.3390/mti7120112
M. Mendoza López, Petronila Mireia Alcaraz Artero, Carlos Truque Díaz, M. Pardo Ríos, J. J. Hernández Morante, Rafael Melendreras Ruíz
This study focuses on the development and assessment of a serious game for health (SGH) aimed at educating children about cardiopulmonary resuscitation (CPR). A video game was created using the Berkeley Snap platform, which uses block programming. Eye-tracking technology was utilized to validate the graphic design. To assess the tool’s effectiveness, a pre-post analytical study was conducted with primary education children to measure the knowledge acquired. The study involved 52 participants with a mean age of 9 years. The results from a custom questionnaire used to measure their theoretical CPR knowledge showed significant improvements in CPR knowledge after the use of the videogame, and their emotional responses improved as well. The assessment of the knowledge acquired through the video game obtained an average score of 5.25 out of 6. Ten video segments consisting of 500 frames each (20 s of video per segment) were analyzed. Within these segments, specific areas that captured the most relevant interaction elements were selected to measure the child’s attention during game play. The average number of gaze fixations, indicating the points in which the child’s attention was placed within the area of interest, was 361.5 out of 500. In conclusion, the utilization of SGH may be an effective method for educating kids about CPR, to provide them with fundamental knowledge relevant to their age group.
本研究的重点是开发和评估一个严肃的健康游戏(SGH),旨在教育儿童心肺复苏(CPR)。使用伯克利Snap平台创建了一个视频游戏,该平台使用块编程。利用眼动追踪技术对平面设计进行验证。为了评估该工具的有效性,对小学教育儿童进行了一项前后分析研究,以衡量获得的知识。这项研究涉及52名平均年龄为9岁的参与者。一份用于测量他们CPR理论知识的定制问卷的结果显示,在使用电子游戏后,他们的CPR知识有了显著提高,他们的情绪反应也有所改善。通过视频游戏获得的知识评估获得了5.25分(满分6分)的平均得分。分析了10个视频片段,每段500帧(每段20秒)。在这些细分中,我们选择了能够捕获最相关互动元素的特定区域来衡量儿童在游戏过程中的注意力。平均注视次数为361.5次(满分500次),这表明孩子的注意力集中在感兴趣的区域内。总之,利用SGH可能是一种有效的方法来教育儿童心肺复苏术,为他们提供与他们年龄组相关的基础知识。
{"title":"Kids Save Lives by Learning through a Serious Game","authors":"M. Mendoza López, Petronila Mireia Alcaraz Artero, Carlos Truque Díaz, M. Pardo Ríos, J. J. Hernández Morante, Rafael Melendreras Ruíz","doi":"10.3390/mti7120112","DOIUrl":"https://doi.org/10.3390/mti7120112","url":null,"abstract":"This study focuses on the development and assessment of a serious game for health (SGH) aimed at educating children about cardiopulmonary resuscitation (CPR). A video game was created using the Berkeley Snap platform, which uses block programming. Eye-tracking technology was utilized to validate the graphic design. To assess the tool’s effectiveness, a pre-post analytical study was conducted with primary education children to measure the knowledge acquired. The study involved 52 participants with a mean age of 9 years. The results from a custom questionnaire used to measure their theoretical CPR knowledge showed significant improvements in CPR knowledge after the use of the videogame, and their emotional responses improved as well. The assessment of the knowledge acquired through the video game obtained an average score of 5.25 out of 6. Ten video segments consisting of 500 frames each (20 s of video per segment) were analyzed. Within these segments, specific areas that captured the most relevant interaction elements were selected to measure the child’s attention during game play. The average number of gaze fixations, indicating the points in which the child’s attention was placed within the area of interest, was 361.5 out of 500. In conclusion, the utilization of SGH may be an effective method for educating kids about CPR, to provide them with fundamental knowledge relevant to their age group.","PeriodicalId":52297,"journal":{"name":"Multimodal Technologies and Interaction","volume":" 6","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138613066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Multimodal Technologies and Interaction
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1