Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00068
H. Aoki
In this study, the creation of pseudo-haptics during swimming in a virtual reality (VR) space. For this purpose, the user can swim in the VR space, to visualize spheres floating around them in the water. The spheres move from the front of the user to the rear as the user perform a breaststroke. Perception of the movement of these spheres can create the sensation of swimming against the flow of water. Thus, the developed system presents pseudo-haptics by controlling the amount of movement of these spheres. Four types of presentation methods were experimentally examined and compared, and their effects were verified by a psychophysical method. The results suggested that the sensation during pseudohaptics can be finely separated into different levels by generating a constant fluid force against the user.
{"title":"Study on Pseudo-haptics during Swimming Motion in a Virtual Reality Space","authors":"H. Aoki","doi":"10.1109/AIVR50618.2020.00068","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00068","url":null,"abstract":"In this study, the creation of pseudo-haptics during swimming in a virtual reality (VR) space. For this purpose, the user can swim in the VR space, to visualize spheres floating around them in the water. The spheres move from the front of the user to the rear as the user perform a breaststroke. Perception of the movement of these spheres can create the sensation of swimming against the flow of water. Thus, the developed system presents pseudo-haptics by controlling the amount of movement of these spheres. Four types of presentation methods were experimentally examined and compared, and their effects were verified by a psychophysical method. The results suggested that the sensation during pseudohaptics can be finely separated into different levels by generating a constant fluid force against the user.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115572993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00026
Mar González-Franco, Zelia Egan, Matt Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Cheng Yao Wang, Derek F. Reilly, Tabitha C. Peck, A. S. Won, A. Steed, E. Ofek
This paper presents MoveBox an open sourced toolbox for animating motion captured (MoCap) movements onto the Microsoft Rocketbox library of avatars. Motion capture is performed using a single depth sensor, such as Azure Kinect or Windows Kinect V2. Motion capture is performed in real-time using a single depth sensor, such as Azure Kinect or Windows Kinect V2, or extracted from existing RGB videos offline leveraging deep-learning computer vision techniques. Our toolbox enables real-time animation of the user’s avatar by converting the transformations between systems that have different joints and hierarchies. Additional features of the toolbox include recording, playback and looping animations, as well as basic audio lip sync, blinking and resizing of avatars as well as finger and hand animations. Our main contribution is both in the creation of this open source tool as well as the validation on different devices and discussion of MoveBox’s capabilities by end users.
{"title":"MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library","authors":"Mar González-Franco, Zelia Egan, Matt Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Cheng Yao Wang, Derek F. Reilly, Tabitha C. Peck, A. S. Won, A. Steed, E. Ofek","doi":"10.1109/AIVR50618.2020.00026","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00026","url":null,"abstract":"This paper presents MoveBox an open sourced toolbox for animating motion captured (MoCap) movements onto the Microsoft Rocketbox library of avatars. Motion capture is performed using a single depth sensor, such as Azure Kinect or Windows Kinect V2. Motion capture is performed in real-time using a single depth sensor, such as Azure Kinect or Windows Kinect V2, or extracted from existing RGB videos offline leveraging deep-learning computer vision techniques. Our toolbox enables real-time animation of the user’s avatar by converting the transformations between systems that have different joints and hierarchies. Additional features of the toolbox include recording, playback and looping animations, as well as basic audio lip sync, blinking and resizing of avatars as well as finger and hand animations. Our main contribution is both in the creation of this open source tool as well as the validation on different devices and discussion of MoveBox’s capabilities by end users.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123938316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Autistic children suffer from distraction and difficulty in learning. The research is still ongoing continues to find suitable ways to help autistic children learn and live normally. Recently, the use of digital technologies in supporting children with Autism has increased dramatically. We focus on problems that autistic children face in the learning process. We propose a new learning system based on Augmented Reality overlie digital objects on top of physical cards and rendering them as a 3D object on mobile devices to help in teaching eating food skills using related phrases and sounds. We aim to improve the learning abilities to repeat the correct behavior.
{"title":"Using Marker Based Augmented Reality to teach autistic eating skills","authors":"Rahma Bouaziz, Maimounah Alhejaili, Raneem Al-Saedi, Abrar Mihdhar, Jawaher Alsarrani","doi":"10.1109/AIVR50618.2020.00050","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00050","url":null,"abstract":"Autistic children suffer from distraction and difficulty in learning. The research is still ongoing continues to find suitable ways to help autistic children learn and live normally. Recently, the use of digital technologies in supporting children with Autism has increased dramatically. We focus on problems that autistic children face in the learning process. We propose a new learning system based on Augmented Reality overlie digital objects on top of physical cards and rendering them as a 3D object on mobile devices to help in teaching eating food skills using related phrases and sounds. We aim to improve the learning abilities to repeat the correct behavior.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"201 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131504646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00054
Costa Anton, Oana Mitrut, A. Moldoveanu, F. Moldoveanu, J. Kosinka
Much of the costs and dangers of exposure therapy in phobia treatment can be removed through virtual reality (VR). Exposing people to heights, for instance, might sound easy, but it still involves time and money investments to reach a tall building, mountain or bridge. People suffering from milder forms of acrophobia might not even be treated at all, the cost not being worth it. This paper presents a prototype that allows exposure therapy to be done in a controlled environment, in a more comfortable, quick and cheaper way. By applying acrophobia questionnaires, collecting biophysical data and developing a virtual reality game, we can expose volunteers to heights and analyze if there is any change in their fear and anxiety levels. This way, regardless of the initial anxiety level and phobia severity, we can check if there is any post-therapy improvement and verify if virtual reality is a viable alternative to real-world exposure.
{"title":"A serious VR game for acrophobia therapy in an urban environment","authors":"Costa Anton, Oana Mitrut, A. Moldoveanu, F. Moldoveanu, J. Kosinka","doi":"10.1109/AIVR50618.2020.00054","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00054","url":null,"abstract":"Much of the costs and dangers of exposure therapy in phobia treatment can be removed through virtual reality (VR). Exposing people to heights, for instance, might sound easy, but it still involves time and money investments to reach a tall building, mountain or bridge. People suffering from milder forms of acrophobia might not even be treated at all, the cost not being worth it. This paper presents a prototype that allows exposure therapy to be done in a controlled environment, in a more comfortable, quick and cheaper way. By applying acrophobia questionnaires, collecting biophysical data and developing a virtual reality game, we can expose volunteers to heights and analyze if there is any change in their fear and anxiety levels. This way, regardless of the initial anxiety level and phobia severity, we can check if there is any post-therapy improvement and verify if virtual reality is a viable alternative to real-world exposure.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128038942","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00023
Salvador Alvídrez, Jorge Peña
The present study analyzes the extent to which verbal mimicry contributes to improving outgroup perceptions in virtual reality (VR) interactions. Particularly, this study examined the interplay between avatar customization, the salience of a common ingroup identity, and verbal mimicry in 54 VR dyads comprising users from different ethnic backgrounds. Participants were asked to customize their avatars to look either like themselves or someone completely different. Participants interacted wearing either similar avatar uniforms (salient common identity) or different clothes (nonsalient identity). The linguistic style matching (LSM) algorithm was employed to calculate verbal mimicry in the communication exchanged during a joint task. The results suggested that verbal mimicry significantly predicted lesser social distance and greater social attraction towards the outgroup member. These results are discussed in terms of their contribution for potential intergroup models of avatar communication in immersive virtual environments (IVEs).
{"title":"Verbal Mimicry Predicts Social Distance and Social Attraction to an Outgroup Member in Virtual Reality","authors":"Salvador Alvídrez, Jorge Peña","doi":"10.1109/AIVR50618.2020.00023","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00023","url":null,"abstract":"The present study analyzes the extent to which verbal mimicry contributes to improving outgroup perceptions in virtual reality (VR) interactions. Particularly, this study examined the interplay between avatar customization, the salience of a common ingroup identity, and verbal mimicry in 54 VR dyads comprising users from different ethnic backgrounds. Participants were asked to customize their avatars to look either like themselves or someone completely different. Participants interacted wearing either similar avatar uniforms (salient common identity) or different clothes (nonsalient identity). The linguistic style matching (LSM) algorithm was employed to calculate verbal mimicry in the communication exchanged during a joint task. The results suggested that verbal mimicry significantly predicted lesser social distance and greater social attraction towards the outgroup member. These results are discussed in terms of their contribution for potential intergroup models of avatar communication in immersive virtual environments (IVEs).","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121091441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00062
Alexander Arntz, S. Eimler, H. Hoppe
The usage of AI enhanced robots in shared task environments is likely to become more and more common with the increase of digitalization in different industrial sectors. To take up this new challenge, research on the design of Human-Robot-Collaboration (HRC) involving AI-based systems has yet to establish common targets and guidelines. This paper presents results from an explorative qualitative study. Participants (N= 80) were either exposed to a virtual representation of an industrial robot-arm equipped with several augmentation channels for communication with the human operator (lights, textual statements about intentions, etc.) or one with no communicative functions at all. Across all conditions, participants recognized the benefit of collaborating with robots in industrial scenarios regarding work efficiency and alleviation of working conditions. However, a communication channel from the robot to the human is crucial for achieving these benefits. Participants interacting with the non-communicative robot expressed dissatisfaction about the workflow. In both conditions we found remarks about the insufficient speed of the robot-arm for an efficient collaborative process. Our results indicate a wider spectrum of questions to be further explored in the design of collaborative experiences with intelligent technological counterparts considering efficiency, safety, economic success and well-being.
{"title":"“The Robot-Arm Talks Back to Me” - Human Perception of Augmented Human-Robot Collaboration in Virtual Reality","authors":"Alexander Arntz, S. Eimler, H. Hoppe","doi":"10.1109/AIVR50618.2020.00062","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00062","url":null,"abstract":"The usage of AI enhanced robots in shared task environments is likely to become more and more common with the increase of digitalization in different industrial sectors. To take up this new challenge, research on the design of Human-Robot-Collaboration (HRC) involving AI-based systems has yet to establish common targets and guidelines. This paper presents results from an explorative qualitative study. Participants (N= 80) were either exposed to a virtual representation of an industrial robot-arm equipped with several augmentation channels for communication with the human operator (lights, textual statements about intentions, etc.) or one with no communicative functions at all. Across all conditions, participants recognized the benefit of collaborating with robots in industrial scenarios regarding work efficiency and alleviation of working conditions. However, a communication channel from the robot to the human is crucial for achieving these benefits. Participants interacting with the non-communicative robot expressed dissatisfaction about the workflow. In both conditions we found remarks about the insufficient speed of the robot-arm for an efficient collaborative process. Our results indicate a wider spectrum of questions to be further explored in the design of collaborative experiences with intelligent technological counterparts considering efficiency, safety, economic success and well-being.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124296729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00048
D. Keßler, Alexander Arntz, J. Friedhoff, S. Eimler
Virtual Reality (VR) holds great potential for new didactic concepts in teaching, since environments, information and objects can be represented and manipulated digitally. Especially when it comes to training environments that include potentially dangerous processes, are expensive or bring the risk of damage to important tools, VR offers an alternative way of approaching a new subject. This paper presents a VR-application used in the studies of mechanical engineering. It includes the virtual representation of a Hermle CNC C42U milling machine, which serves to acquire basic knowledge in controlling such a system, avoiding safety risks and logistical constraints. Results from an evaluation with the target group show a good usability and (perceived) impact on the user’s learning gain.
{"title":"Mill Instructor: Teaching Industrial CNC Procedures Using Virtual Reality","authors":"D. Keßler, Alexander Arntz, J. Friedhoff, S. Eimler","doi":"10.1109/AIVR50618.2020.00048","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00048","url":null,"abstract":"Virtual Reality (VR) holds great potential for new didactic concepts in teaching, since environments, information and objects can be represented and manipulated digitally. Especially when it comes to training environments that include potentially dangerous processes, are expensive or bring the risk of damage to important tools, VR offers an alternative way of approaching a new subject. This paper presents a VR-application used in the studies of mechanical engineering. It includes the virtual representation of a Hermle CNC C42U milling machine, which serves to acquire basic knowledge in controlling such a system, avoiding safety risks and logistical constraints. Results from an evaluation with the target group show a good usability and (perceived) impact on the user’s learning gain.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"1994 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125544153","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00034
Gang Li, Mark Mcgill, S. Brewster, F. Pollick
With the development of consumer virtual reality (VR), people have increasing opportunities to experience cybersickness (CS) –- a kind of visuallyinduced motion sickness (MS). In view of the importance of CS mitigation (CSM), this paper reviews the methods of electrostimulation-based CSM (e-CSM), broadly categorised as either “VR-centric” or “Human-centric”. “VR-centric” refers to approaches where knowledge regarding the visual motion being experienced in VR directly affects how the neurostimulation is delivered, whereas “Human-centric” approaches focus on the inhibition or enhancement of human functions per se without knowledge of the experienced visual motion. We DIFFERENT E-found that 1) most e-CSM approaches are based on visual-vestibular sensory conflict theory –- one of the generally-accepted aetiologies of MS, 2) the majority of eCSM approaches are vestibular system-centric, either stimulating it to compensate for the mismatched vestibular sensory responses, or inhibiting it to make an artificial and temporary dysfunction in vestibular sensory organs or cortical areas, 3) Vestibular sensory organbased solutions are able to mitigate CS with immediate effect, while the real-time effect of vestibular cortical areas-based methods remains unclear, due to limited public data, 4) Based on subjective assessment, VRcentric approaches could relieve all three kinds of symptoms (nausea, oculomotor, and disorientation), which appears superior to the human-centric ones that could only alleviate one of the symptom types or just have an overall relief effect. Finally, we propose promising future research directions in the development of e-CSM.
{"title":"A Review of Electrostimulation-based Cybersickness Mitigations","authors":"Gang Li, Mark Mcgill, S. Brewster, F. Pollick","doi":"10.1109/AIVR50618.2020.00034","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00034","url":null,"abstract":"With the development of consumer virtual reality (VR), people have increasing opportunities to experience cybersickness (CS) –- a kind of visuallyinduced motion sickness (MS). In view of the importance of CS mitigation (CSM), this paper reviews the methods of electrostimulation-based CSM (e-CSM), broadly categorised as either “VR-centric” or “Human-centric”. “VR-centric” refers to approaches where knowledge regarding the visual motion being experienced in VR directly affects how the neurostimulation is delivered, whereas “Human-centric” approaches focus on the inhibition or enhancement of human functions per se without knowledge of the experienced visual motion. We DIFFERENT E-found that 1) most e-CSM approaches are based on visual-vestibular sensory conflict theory –- one of the generally-accepted aetiologies of MS, 2) the majority of eCSM approaches are vestibular system-centric, either stimulating it to compensate for the mismatched vestibular sensory responses, or inhibiting it to make an artificial and temporary dysfunction in vestibular sensory organs or cortical areas, 3) Vestibular sensory organbased solutions are able to mitigate CS with immediate effect, while the real-time effect of vestibular cortical areas-based methods remains unclear, due to limited public data, 4) Based on subjective assessment, VRcentric approaches could relieve all three kinds of symptoms (nausea, oculomotor, and disorientation), which appears superior to the human-centric ones that could only alleviate one of the symptom types or just have an overall relief effect. Finally, we propose promising future research directions in the development of e-CSM.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129335365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00064
Alexander Arntz, S. Eimler, D. Keßler, A. Nabokova, S. Schädlich
Augmented Reality (AR) has great potential for new didactic concepts in teaching. Environments, information and objects can be comprehensively and dynamically represented, supporting self-paced and holistic learning. This paper presents an implementation of a multimodal AR-application for the purpose of teaching complex features and mechanics of a ”Heating, Ventilation and Air Conditioning System” in a situated and engaging way. The application was designed and implemented by an interdisciplinary team and evaluated in a mixed-methods approach. Results show a high usability and acceptance of the application. Students recognized the benefit of the application regarding their motivation and learning gains and made suggestions for further improvements.
{"title":"Thermodynamics Reloaded: Experiencing Heating, Ventilation and Air Conditioning in AR","authors":"Alexander Arntz, S. Eimler, D. Keßler, A. Nabokova, S. Schädlich","doi":"10.1109/AIVR50618.2020.00064","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00064","url":null,"abstract":"Augmented Reality (AR) has great potential for new didactic concepts in teaching. Environments, information and objects can be comprehensively and dynamically represented, supporting self-paced and holistic learning. This paper presents an implementation of a multimodal AR-application for the purpose of teaching complex features and mechanics of a ”Heating, Ventilation and Air Conditioning System” in a situated and engaging way. The application was designed and implemented by an interdisciplinary team and evaluated in a mixed-methods approach. Results show a high usability and acceptance of the application. Students recognized the benefit of the application regarding their motivation and learning gains and made suggestions for further improvements.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115049126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-06DOI: 10.1109/AIVR50618.2020.00025
Philipp Ladwig, Alexander Pech, R. Dörner, C. Geiger
Face-to-face conversation in Virtual Reality (VR) is a challenge when participants wear head-mounted displays (HMD). A significant portion of a participant’s face is hidden and facial expressions are difficult to perceive. Past research has shown that high-fidelity face reconstruction with personal avatars in VR is possible under laboratory conditions with high-cost hardware. In this paper, we propose one of the first low-cost systems for this task which uses only open source, free software and affordable hardware. Our approach is to track the user’s face underneath the HMD utilizing a Convolutional Neural Network (CNN) and generate corresponding expressions with Generative Adversarial Networks (GAN) for producing RGBD images of the person’s face. We use commodity hardware with low-cost extensions such as 3Dprinted mounts and miniature cameras. Our approach learns end-to-end without manual intervention, runs in real time, and can be trained and executed on an ordinary gaming computer. We report evaluation results showing that our low-cost system does not achieve the same fidelity of research prototypes using high-end hardware and closed source software, but it is capable of creating individual facial avatars with personspecific characteristics in movements and expressions.
{"title":"Unmasking Communication Partners: A Low-Cost AI Solution for Digitally Removing Head-Mounted Displays in VR-Based Telepresence","authors":"Philipp Ladwig, Alexander Pech, R. Dörner, C. Geiger","doi":"10.1109/AIVR50618.2020.00025","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00025","url":null,"abstract":"Face-to-face conversation in Virtual Reality (VR) is a challenge when participants wear head-mounted displays (HMD). A significant portion of a participant’s face is hidden and facial expressions are difficult to perceive. Past research has shown that high-fidelity face reconstruction with personal avatars in VR is possible under laboratory conditions with high-cost hardware. In this paper, we propose one of the first low-cost systems for this task which uses only open source, free software and affordable hardware. Our approach is to track the user’s face underneath the HMD utilizing a Convolutional Neural Network (CNN) and generate corresponding expressions with Generative Adversarial Networks (GAN) for producing RGBD images of the person’s face. We use commodity hardware with low-cost extensions such as 3Dprinted mounts and miniature cameras. Our approach learns end-to-end without manual intervention, runs in real time, and can be trained and executed on an ordinary gaming computer. We report evaluation results showing that our low-cost system does not achieve the same fidelity of research prototypes using high-end hardware and closed source software, but it is capable of creating individual facial avatars with personspecific characteristics in movements and expressions.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131103092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：