This paper improves the accuracy of heatmap-based 3D face alignment neural networks. Many current approaches in face alignment are limited by two major problems, quantization and the lack of regularization of heatmaps. The first limitation is caused by the non-differentiable argmax function, which extracts landmark coordinates from heatmaps as integer indices. Heatmaps are generated at low-resolution to reduce the memory and computational costs, which results in heatmaps far lower than the input image’s resolution. We propose a heatmap generator network producing floating-point precision heatmaps that are scalable to higher-resolutions. To resolve the second limitation, we propose a novel deformation constraint on heatmaps. The constraint is based on graph-Laplacian and enables a heatmap generator to regularize overall shape of the output face landmarks using the global face structure. By eliminating quantization and including regularization, our method can vastly improve landmark localization accuracy, and achieves the state-of-the-art performance without adding complex network structures.
{"title":"Floating-point Precision and Deformation Awareness for Scalable and Robust 3D Face Alignment","authors":"Jacob Morton, Seungyong Lee","doi":"10.1145/3359996.3364260","DOIUrl":"https://doi.org/10.1145/3359996.3364260","url":null,"abstract":"This paper improves the accuracy of heatmap-based 3D face alignment neural networks. Many current approaches in face alignment are limited by two major problems, quantization and the lack of regularization of heatmaps. The first limitation is caused by the non-differentiable argmax function, which extracts landmark coordinates from heatmaps as integer indices. Heatmaps are generated at low-resolution to reduce the memory and computational costs, which results in heatmaps far lower than the input image’s resolution. We propose a heatmap generator network producing floating-point precision heatmaps that are scalable to higher-resolutions. To resolve the second limitation, we propose a novel deformation constraint on heatmaps. The constraint is based on graph-Laplacian and enables a heatmap generator to regularize overall shape of the output face landmarks using the global face structure. By eliminating quantization and including regularization, our method can vastly improve landmark localization accuracy, and achieves the state-of-the-art performance without adding complex network structures.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131003163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mayra Donaji Barrera Machuca, W. Stuerzlinger, P. Asente
Most current commercial Virtual Reality (VR) drawing applications for creativity rely on freehand 3D drawing as their main interaction paradigm. However, the presence of the additional third dimension makes accurate freehand drawing challenging. Some systems address this problem by constraining or beautifying user strokes, which can be intrusive and can limit the expressivity of freehand drawing. In this paper, we evaluate the effectiveness of relying solely on visual guidance to increase overall drawing shape-likeness. We identified a set of common mistakes that users make while creating freehand strokes in VR and then designed a set of visual guides, the Smart3DGuides, which help users avoid these mistakes. We evaluated Smart3DGuides in two user studies, and our results show that non-constraining visual guides help users draw more accurately.
{"title":"Smart3DGuides: Making Unconstrained Immersive 3D Drawing More Accurate","authors":"Mayra Donaji Barrera Machuca, W. Stuerzlinger, P. Asente","doi":"10.1145/3359996.3364254","DOIUrl":"https://doi.org/10.1145/3359996.3364254","url":null,"abstract":"Most current commercial Virtual Reality (VR) drawing applications for creativity rely on freehand 3D drawing as their main interaction paradigm. However, the presence of the additional third dimension makes accurate freehand drawing challenging. Some systems address this problem by constraining or beautifying user strokes, which can be intrusive and can limit the expressivity of freehand drawing. In this paper, we evaluate the effectiveness of relying solely on visual guidance to increase overall drawing shape-likeness. We identified a set of common mistakes that users make while creating freehand strokes in VR and then designed a set of visual guides, the Smart3DGuides, which help users avoid these mistakes. We evaluated Smart3DGuides in two user studies, and our results show that non-constraining visual guides help users draw more accurately.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116446549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A key challenge to effective storytelling using Virtual Reality (VR), such as with 360-degree videos, is how to direct user attention to important content without taking away user agency for free exploration. In this paper, we introduce the notion of an Action Unit system, composed of social cues such as head and arm movements, as a way of directing users to focus on content important for the given narrative. We applied this idea to a 360-degree VR tour, and evaluated its effects on memory, engagement, enjoyment, and cyber-sickness. The results indicate that the levels of engagement and enjoyment increased when these Action Units were applied. Users also preferred the Action Units for their diegetic aspects.
{"title":"Action Units: Directing User Attention in 360-degree Video based VR","authors":"Lingwei Tong, Sungchul Jung, R. Lindeman","doi":"10.1145/3359996.3364706","DOIUrl":"https://doi.org/10.1145/3359996.3364706","url":null,"abstract":"A key challenge to effective storytelling using Virtual Reality (VR), such as with 360-degree videos, is how to direct user attention to important content without taking away user agency for free exploration. In this paper, we introduce the notion of an Action Unit system, composed of social cues such as head and arm movements, as a way of directing users to focus on content important for the given narrative. We applied this idea to a 360-degree VR tour, and evaluated its effects on memory, engagement, enjoyment, and cyber-sickness. The results indicate that the levels of engagement and enjoyment increased when these Action Units were applied. Users also preferred the Action Units for their diegetic aspects.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128331311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents an obstacle detection and alert system for the pedestrians who use smartphone AR applications. The system analyzes the input camera image to extract feature points and determines whether the feature points come from obstacles ahead in the path. With the obstacle detector, two experiments were made. The first investigated the obstacle alert interfaces, and the second investigated the orientation guide interfaces that instruct users to hold their smartphones with some angles/orientations appropriate to capture the environment. Then, the best interfaces identified from the experiments were integrated and tested to examine their usability and user experiences.
{"title":"Obstacle Detection and Alert System for Smartphone AR Users","authors":"Hyeongyeop Kang, Geonsun Lee, Junghyun Han","doi":"10.1145/3359996.3364256","DOIUrl":"https://doi.org/10.1145/3359996.3364256","url":null,"abstract":"This paper presents an obstacle detection and alert system for the pedestrians who use smartphone AR applications. The system analyzes the input camera image to extract feature points and determines whether the feature points come from obstacles ahead in the path. With the obstacle detector, two experiments were made. The first investigated the obstacle alert interfaces, and the second investigated the orientation guide interfaces that instruct users to hold their smartphones with some angles/orientations appropriate to capture the environment. Then, the best interfaces identified from the experiments were integrated and tested to examine their usability and user experiences.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127191622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose a concept of a novel interaction strategy for providing rich haptic feedback in Virtual Reality (VR), when each user’s finger is connected to micro-quadrotor with a wire. Described technology represents the first flying wearable haptic interface. The solution potentially is able to deliver high resolution force feedback to each finger during fine motor interaction in VR. The tips of tethers are connected to the centers of quadcopters under their bottom. Therefore, flight stability is increasing and the interaction forces are becoming stronger which allows to use smaller drones.
{"title":"WiredSwarm: High Resolution Haptic Feedback Provided by a Swarm of Drones to the User’s Fingers for VR interaction","authors":"Evgeny V. Tsykunov, D. Tsetserukou","doi":"10.1145/3359996.3364789","DOIUrl":"https://doi.org/10.1145/3359996.3364789","url":null,"abstract":"We propose a concept of a novel interaction strategy for providing rich haptic feedback in Virtual Reality (VR), when each user’s finger is connected to micro-quadrotor with a wire. Described technology represents the first flying wearable haptic interface. The solution potentially is able to deliver high resolution force feedback to each finger during fine motor interaction in VR. The tips of tethers are connected to the centers of quadcopters under their bottom. Therefore, flight stability is increasing and the interaction forces are becoming stronger which allows to use smaller drones.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123735906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Powdered materials, such as sand and flour, are quite common in nature, whose properties always range from granular particles to smog materials under the air friction while throwing. This paper presents a hybrid method that tightly couples APIC solver with density field to accomplish the transformation of continuous powdered materials varying among granular particles, smog, powders and their natural mixtures. In our method, a part of the granular particles will be transformed to dust smog while interacting with air and represented by density field, then, as velocity decreases the density-based dust will deposit to powder particles. We construct a unified framework to imitate the mutual transformation process for the powdered materials of different scales, which greatly enhance the details of particle-based materials modeling. We have conducted extensive experiments to verify the performance of our model, and get satisfactory results in terms of stability, efficiency and visual authenticity as expected.
{"title":"A Hybrid Method for Powdered Materials Modeling","authors":"Yang Gao, Yinghao Xu, Shuai Li, A. Hao, Hong Qin","doi":"10.1145/3359996.3364266","DOIUrl":"https://doi.org/10.1145/3359996.3364266","url":null,"abstract":"Powdered materials, such as sand and flour, are quite common in nature, whose properties always range from granular particles to smog materials under the air friction while throwing. This paper presents a hybrid method that tightly couples APIC solver with density field to accomplish the transformation of continuous powdered materials varying among granular particles, smog, powders and their natural mixtures. In our method, a part of the granular particles will be transformed to dust smog while interacting with air and represented by density field, then, as velocity decreases the density-based dust will deposit to powder particles. We construct a unified framework to imitate the mutual transformation process for the powdered materials of different scales, which greatly enhance the details of particle-based materials modeling. We have conducted extensive experiments to verify the performance of our model, and get satisfactory results in terms of stability, efficiency and visual authenticity as expected.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130612881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a prototype mobile augmented reality interface for assisting the process of learning folk dances. As a case study, a folk dance was digitized based on recordings from professional dancers. To assess the effectiveness of the technology, it was comparatively evaluated with a large back-projection system in laboratory conditions. Sixteen participants took part in the study, and their movements were captured using motion capture system and then compared with the recordings from the professional dancers. Experimental results indicate that augmented reality has the potential to be used for learning folk dances.
{"title":"A Mobile Augmented Reality Interface for Teaching Folk Dances","authors":"Iris Kico, F. Liarokapis","doi":"10.1145/3359996.3364752","DOIUrl":"https://doi.org/10.1145/3359996.3364752","url":null,"abstract":"This paper presents a prototype mobile augmented reality interface for assisting the process of learning folk dances. As a case study, a folk dance was digitized based on recordings from professional dancers. To assess the effectiveness of the technology, it was comparatively evaluated with a large back-projection system in laboratory conditions. Sixteen participants took part in the study, and their movements were captured using motion capture system and then compared with the recordings from the professional dancers. Experimental results indicate that augmented reality has the potential to be used for learning folk dances.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131294164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. K. D. Anjos, Maurício Sousa, Daniel Mendes, Daniel Medeiros, M. Billinghurst, C. Anslow, Joaquim Jorge
Modern volumetric projection-based telepresence approaches are capable of providing realistic full-size virtual representations of remote people. Interacting with full-size people may not be desirable due to the spatial constraints of the physical environment, application context, or display technology. However, the miniaturization of remote people is known to create an eye gaze matching problem. Eye-contact is essential to communication as it allows for people to use natural nonverbal cues and improves the sense of “being there”. In this paper we discuss the design space for interacting with volumetric representations of people and present an approach for dynamically manipulating scale, orientation and the position of holograms which guarantees eye-contact. We created a working augmented reality-based prototype and validated it with 14 participants.
{"title":"Adventures in Hologram Space: Exploring the Design Space of Eye-to-eye Volumetric Telepresence","authors":"R. K. D. Anjos, Maurício Sousa, Daniel Mendes, Daniel Medeiros, M. Billinghurst, C. Anslow, Joaquim Jorge","doi":"10.1145/3359996.3364244","DOIUrl":"https://doi.org/10.1145/3359996.3364244","url":null,"abstract":"Modern volumetric projection-based telepresence approaches are capable of providing realistic full-size virtual representations of remote people. Interacting with full-size people may not be desirable due to the spatial constraints of the physical environment, application context, or display technology. However, the miniaturization of remote people is known to create an eye gaze matching problem. Eye-contact is essential to communication as it allows for people to use natural nonverbal cues and improves the sense of “being there”. In this paper we discuss the design space for interacting with volumetric representations of people and present an approach for dynamically manipulating scale, orientation and the position of holograms which guarantees eye-contact. We created a working augmented reality-based prototype and validated it with 14 participants.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131313680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study investigated whether individual differences in postural stability/activity can be used to predict who will become sick when exposed to head-mounted display (HMD) based virtual reality (VR). We found that participants who reported feeling sick after at least one exposure to VR displayed different postural activity than those who remained well. Importantly these differences were present in their sway data before they even donned the HMD. These results are inline with the postural instability theory of motion sickness and suggest that we can identify individuals who are more susceptible HMD-based cybersickness based on their spontaneous postural sway.
{"title":"Can We Predict Susceptibility to Cybersickness?","authors":"Dante Risi, S. Palmisano","doi":"10.1145/3359996.3364705","DOIUrl":"https://doi.org/10.1145/3359996.3364705","url":null,"abstract":"This study investigated whether individual differences in postural stability/activity can be used to predict who will become sick when exposed to head-mounted display (HMD) based virtual reality (VR). We found that participants who reported feeling sick after at least one exposure to VR displayed different postural activity than those who remained well. Importantly these differences were present in their sway data before they even donned the HMD. These results are inline with the postural instability theory of motion sickness and suggest that we can identify individuals who are more susceptible HMD-based cybersickness based on their spontaneous postural sway.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128873413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper describes aspects which are important for camera positioning in cinematic virtual reality. For our findings, we took a closer look at proxemics, the study on how humans behave in regard to space and distances. We explored well-known shot sizes used in traditional filmmaking and put them in relation to proxemics distances. The results were adapted to camera distances in cinematic virtual reality.
{"title":"Where to Place the Camera","authors":"Sylvia Rothe, Vanessa Sarakiotis, H. Hussmann","doi":"10.1145/3359996.3364798","DOIUrl":"https://doi.org/10.1145/3359996.3364798","url":null,"abstract":"This paper describes aspects which are important for camera positioning in cinematic virtual reality. For our findings, we took a closer look at proxemics, the study on how humans behave in regard to space and distances. We explored well-known shot sizes used in traditional filmmaking and put them in relation to proxemics distances. The results were adapted to camera distances in cinematic virtual reality.","PeriodicalId":393864,"journal":{"name":"Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2019-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125529250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：