Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460059
Max Ehrlich, Philippos Mordohai
We present a novel hand localization technique for 3D user interfaces. Our method is designed to overcome the difficulty of fitting anatomical models which fail to converge or converge with large errors in complex scenes or suboptimal imagery. We learn a discriminative model of the hand from depth images by using fast to compute features and a Random Forest classifier. The learned model is then combined with a spatial clustering algorithm to localize the hand position. We propose three formulations of low-level image features for use in model training. We evaluate the performance of our method by testing on low resolution depth maps of users two to three meters from the sensor in natural poses. Our method can detect an arbitrary number of hands per scene and preliminary results show that it is robust to suboptimal imagery.
{"title":"Discriminative hand localization in depth images","authors":"Max Ehrlich, Philippos Mordohai","doi":"10.1109/3DUI.2016.7460059","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460059","url":null,"abstract":"We present a novel hand localization technique for 3D user interfaces. Our method is designed to overcome the difficulty of fitting anatomical models which fail to converge or converge with large errors in complex scenes or suboptimal imagery. We learn a discriminative model of the hand from depth images by using fast to compute features and a Random Forest classifier. The learned model is then combined with a spatial clustering algorithm to localize the hand position. We propose three formulations of low-level image features for use in model training. We evaluate the performance of our method by testing on low resolution depth maps of users two to three meters from the sensor in natural poses. Our method can detect an arbitrary number of hands per scene and preliminary results show that it is robust to suboptimal imagery.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134289308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.3929/ETHZ-A-010613910
Markus Zank, A. Kunz
Model predictive control was shown to be a powerful tool for Redirected Walking when used to plan and select future redirection techniques. However, to use it effectively, a good prediction of the user's future actions is crucial. Traditionally, this prediction is made based on the user's position or current direction of movement. In the area of cognitive sciences however, it was shown that a person's gaze can also be highly indicative of his intention in both selection and navigation tasks. In this paper, this effect is used the first time to predict a user's locomotion target during goal-directed locomotion in an immersive virtual environment. After discussing the general implications and challenges of using eye tracking for prediction in a locomotion context, we propose a prediction method for a user's intended locomotion target. This approach is then compared with position based approaches in terms of prediction time and accuracy based on data gathered in an experiment. The results show that, in certain situations, eye tracking allows an earlier prediction compared approaches currently used for redirected walking. However, other recently published prediction methods that are based on the user's position perform almost as well as the eye tracking based approaches presented in this paper.
{"title":"Eye tracking for locomotion prediction in redirected walking","authors":"Markus Zank, A. Kunz","doi":"10.3929/ETHZ-A-010613910","DOIUrl":"https://doi.org/10.3929/ETHZ-A-010613910","url":null,"abstract":"Model predictive control was shown to be a powerful tool for Redirected Walking when used to plan and select future redirection techniques. However, to use it effectively, a good prediction of the user's future actions is crucial. Traditionally, this prediction is made based on the user's position or current direction of movement. In the area of cognitive sciences however, it was shown that a person's gaze can also be highly indicative of his intention in both selection and navigation tasks. In this paper, this effect is used the first time to predict a user's locomotion target during goal-directed locomotion in an immersive virtual environment. After discussing the general implications and challenges of using eye tracking for prediction in a locomotion context, we propose a prediction method for a user's intended locomotion target. This approach is then compared with position based approaches in terms of prediction time and accuracy based on data gathered in an experiment. The results show that, in certain situations, eye tracking allows an earlier prediction compared approaches currently used for redirected walking. However, other recently published prediction methods that are based on the user's position perform almost as well as the eye tracking based approaches presented in this paper.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134642958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460029
R. Tanaka, Takuji Narumi, T. Tanikawa, M. Hirose
It is known that interacting with virtual objects enhances users' understanding and interests more effectively than learning through passive media. However, compared with the passive media, the excessive amount of information and interactive options in most virtual reality settings may cause users to quit exploring before they experience the entire content of the virtual world. In this paper, we propose a new guidance method to implicitly lead users to pre-defined locations in the virtual environment while continuing to permit free explorations by using a kind of potential field (guidance field). The guidance field is composed of two independent mechanisms: locomotion guidance and rotation guidance. We implemented our method in a virtual museum exploring system and exhibited it in a real museum to evaluate the effectiveness of our method when used by a large number of people. The result suggests that our method successfully guides users to pre-defined locations and makes users aware of pre-defined objects. Moreover, the results suggest that our guidance may not interfere users' arbitrary explorations.
{"title":"Guidance field: Potential field to guide users to target locations in virtual environments","authors":"R. Tanaka, Takuji Narumi, T. Tanikawa, M. Hirose","doi":"10.1109/3DUI.2016.7460029","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460029","url":null,"abstract":"It is known that interacting with virtual objects enhances users' understanding and interests more effectively than learning through passive media. However, compared with the passive media, the excessive amount of information and interactive options in most virtual reality settings may cause users to quit exploring before they experience the entire content of the virtual world. In this paper, we propose a new guidance method to implicitly lead users to pre-defined locations in the virtual environment while continuing to permit free explorations by using a kind of potential field (guidance field). The guidance field is composed of two independent mechanisms: locomotion guidance and rotation guidance. We implemented our method in a virtual museum exploring system and exhibited it in a real museum to evaluate the effectiveness of our method when used by a large number of people. The result suggests that our method successfully guides users to pre-defined locations and makes users aware of pre-defined objects. Moreover, the results suggest that our guidance may not interfere users' arbitrary explorations.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133932936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460072
Sharif Shahnewaz, Imtiaz Afarat, Tanvir Irfan, G. Samaraweera, Mikael Dallaire-Cote, David R. Labbé, J. Quarles
The long term objective of this research is to improve gait (i.e., walking patterns) rehabilitation through the use of 3D user interfaces and virtual embodiment. Previous research has shown that virtual embodiment can elicit behavioral change and increased motivation for exercise. However, there has been minimal research on how virtual embodiment can affect persons undergoing physical rehabilitation. To enable the future study of this, we present Gaitzilla a novel gait rehabilitation game in which the user embodies a gigantic monster who is being attacked by small tanks on the ground. The user must step on the tanks to survive. The required movements in the game are inspired by real gait training exercises that focus on foot placement and control. We utilize 3D user interfaces for control of the user's avatar. This poster presents the concept and implementation of the game, the methodology behind the design, and future considerations for studying the effects of virtual embodiment on gait rehabilitation.
{"title":"Gaitzilla: A game to study the effects of virtual embodiment in gait rehabilitation","authors":"Sharif Shahnewaz, Imtiaz Afarat, Tanvir Irfan, G. Samaraweera, Mikael Dallaire-Cote, David R. Labbé, J. Quarles","doi":"10.1109/3DUI.2016.7460072","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460072","url":null,"abstract":"The long term objective of this research is to improve gait (i.e., walking patterns) rehabilitation through the use of 3D user interfaces and virtual embodiment. Previous research has shown that virtual embodiment can elicit behavioral change and increased motivation for exercise. However, there has been minimal research on how virtual embodiment can affect persons undergoing physical rehabilitation. To enable the future study of this, we present Gaitzilla a novel gait rehabilitation game in which the user embodies a gigantic monster who is being attacked by small tanks on the ground. The user must step on the tanks to survive. The required movements in the game are inspired by real gait training exercises that focus on foot placement and control. We utilize 3D user interfaces for control of the user's avatar. This poster presents the concept and implementation of the game, the methodology behind the design, and future considerations for studying the effects of virtual embodiment on gait rehabilitation.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129030583","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460052
Sabah Boustila, Antonio Capobianco, D. Bechmann, Olivier Génevaux
In this paper, we suggest using a hybrid projection to increase the vertical geometric field of view without incurring large deformations to preserve distance perception and to allow the seeing of the surrounding ground. We have conducted an experiment in furnished and unfurnished houses to evaluate the perception of distances and the spatial comprehension. Results show that the hybrid projection improves the perception of surrounding ground which leads to an improvement in the spatial comprehension. Moreover, it preserves the perception of distances and sizes by providing a performance similar to the perspective one in the task of distance estimation.
{"title":"A hybrid projection to widen the vertical field of view with large screens to improve the perception of personal space in architectural project review","authors":"Sabah Boustila, Antonio Capobianco, D. Bechmann, Olivier Génevaux","doi":"10.1109/3DUI.2016.7460052","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460052","url":null,"abstract":"In this paper, we suggest using a hybrid projection to increase the vertical geometric field of view without incurring large deformations to preserve distance perception and to allow the seeing of the surrounding ground. We have conducted an experiment in furnished and unfurnished houses to evaluate the perception of distances and the spatial comprehension. Results show that the hybrid projection improves the perception of surrounding ground which leads to an improvement in the spatial comprehension. Moreover, it preserves the perception of distances and sizes by providing a performance similar to the perspective one in the task of distance estimation.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121315206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460070
Hitomi Matsuki, Shohei Mori, Sei Ikeda, F. Shibata, Asako Kimura, H. Tamura
In this paper, we introduce novel problems of binocular stereo (binocular mismatching) in observation-based diminished reality. To confirm these problems, we simulated an observation-based diminished reality system using a video see-through head-mounted display. We also demonstrated that simple methods can reduce such binocular mismatching.
{"title":"Considerations on binocular mismatching in observation-based diminished reality","authors":"Hitomi Matsuki, Shohei Mori, Sei Ikeda, F. Shibata, Asako Kimura, H. Tamura","doi":"10.1109/3DUI.2016.7460070","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460070","url":null,"abstract":"In this paper, we introduce novel problems of binocular stereo (binocular mismatching) in observation-based diminished reality. To confirm these problems, we simulated an observation-based diminished reality system using a video see-through head-mounted display. We also demonstrated that simple methods can reduce such binocular mismatching.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128355616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460031
R. Tanaka, Takuji Narumi, T. Tanikawa, M. Hirose
In this paper, we propose Motive Compass, a new navigation interface for locomotion in virtual environment constructed with spherical images. In these virtual environments, users can gaze in all directions, but can only pass through lines where the camera has passed through during recording. Because the rotation of a virtual camera is not constrained to navigation paths, users may perceive that accessible locations are not constrained and they can freely move. Therefore, it is necessary to inform users of the accessible directions. Furthermore, velocity control is needed for close exploration in virtual environments. Therefore, we propose the Motive Compass input interface, which intuitively shows users the accessible directions and enables them to control the velocity. We conducted a large-scale demonstration experiment in a real exhibition to evaluate our interface and compared it with a conventional interface. The results show that our interface more effectively presents accessible directions and enables users to control their velocities, which supports the user's exploration in virtual environments.
{"title":"Motive compass: Navigation interface for locomotion in virtual environments constructed with spherical images","authors":"R. Tanaka, Takuji Narumi, T. Tanikawa, M. Hirose","doi":"10.1109/3DUI.2016.7460031","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460031","url":null,"abstract":"In this paper, we propose Motive Compass, a new navigation interface for locomotion in virtual environment constructed with spherical images. In these virtual environments, users can gaze in all directions, but can only pass through lines where the camera has passed through during recording. Because the rotation of a virtual camera is not constrained to navigation paths, users may perceive that accessible locations are not constrained and they can freely move. Therefore, it is necessary to inform users of the accessible directions. Furthermore, velocity control is needed for close exploration in virtual environments. Therefore, we propose the Motive Compass input interface, which intuitively shows users the accessible directions and enables them to control the velocity. We conducted a large-scale demonstration experiment in a real exhibition to evaluate our interface and compared it with a conventional interface. The results show that our interface more effectively presents accessible directions and enables users to control their velocities, which supports the user's exploration in virtual environments.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124048162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460047
Kenneth R. Moser, J. Edward Swan
Advances in optical see-through head-mounted display technology have yielded a number of consumer accessible options, such as the Google Glass and Epson Moverio BT-200, and have paved the way for promising next generation hardware, including the Microsoft HoloLens and Epson Pro BT-2000. The release of consumer devices, though, has also been accompanied by an ever increasing need for standardized optical see-through display calibration procedures easily implemented and performed by researchers, developers, and novice users alike. Automatic calibration techniques offer the possibility for ubiquitous environment independent solutions, un-reliant upon user interaction. These processes, however, require the use of additional eye tracking hardware and algorithms not natively present in current display offerings. User dependent approaches, therefore, remain the only viable option for effective calibration of current generation optical see-through hardware. Inclusion of depth sensors and hand tracking cameras, promised in forthcoming consumer models, offer further potential to improve these manual methods and provide practical intuitive calibration options accessible to a wide user base. In this work, we evaluate the accuracy and precision of manual optical see-through head-mounted display calibration performed using a Leap Motion controller. Both hand and stylus based methods for monocular and stereo procedures are examined, along with several on-screen reticle designs for improving alignment context during calibration. Our study shows, that while enhancing the context of reticles for hand based alignments does yield improved results, Leap Motion calibrations performed with a stylus offer the most accurate and consistent performance, comparable to that found in previous studies for environment-centric routines. In addition, we found that stereo calibration further improved precision in every case. We believe that our findings not only validate the potential of hand and gesture based trackers in facilitating optical see-through calibration methodologies, but also provide a suitable benchmark to help guide future efforts in standardizing calibration practices for user friendly consumer systems.
{"title":"Evaluation of user-centric optical see-through head-mounted display calibration using a leap motion controller","authors":"Kenneth R. Moser, J. Edward Swan","doi":"10.1109/3DUI.2016.7460047","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460047","url":null,"abstract":"Advances in optical see-through head-mounted display technology have yielded a number of consumer accessible options, such as the Google Glass and Epson Moverio BT-200, and have paved the way for promising next generation hardware, including the Microsoft HoloLens and Epson Pro BT-2000. The release of consumer devices, though, has also been accompanied by an ever increasing need for standardized optical see-through display calibration procedures easily implemented and performed by researchers, developers, and novice users alike. Automatic calibration techniques offer the possibility for ubiquitous environment independent solutions, un-reliant upon user interaction. These processes, however, require the use of additional eye tracking hardware and algorithms not natively present in current display offerings. User dependent approaches, therefore, remain the only viable option for effective calibration of current generation optical see-through hardware. Inclusion of depth sensors and hand tracking cameras, promised in forthcoming consumer models, offer further potential to improve these manual methods and provide practical intuitive calibration options accessible to a wide user base. In this work, we evaluate the accuracy and precision of manual optical see-through head-mounted display calibration performed using a Leap Motion controller. Both hand and stylus based methods for monocular and stereo procedures are examined, along with several on-screen reticle designs for improving alignment context during calibration. Our study shows, that while enhancing the context of reticles for hand based alignments does yield improved results, Leap Motion calibrations performed with a stylus offer the most accurate and consistent performance, comparable to that found in previous studies for environment-centric routines. In addition, we found that stereo calibration further improved precision in every case. We believe that our findings not only validate the potential of hand and gesture based trackers in facilitating optical see-through calibration methodologies, but also provide a suitable benchmark to help guide future efforts in standardizing calibration practices for user friendly consumer systems.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115448417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460071
Jérémy Plouzeau, Aida Erfanian, Cynthia Chiu, F. Mérienne, Yaoping Hu
In this study, we present a preliminary exploration about the added value of vibration information for guiding navigation in a VE. The exploration consists of two parts. Firstly, we designed two different vibration patterns. These patterns, pushing pattern and compass pattern, differ conceptually in the levels of abstraction. Secondly, we undertook an experiment to compare the two patterns in guiding navigation in a VE. The objective of the comparison is to establish a baseline for examining the suitability of using vibration patterns to guide navigation.
{"title":"Navigation in virtual environments: Design and comparison of two anklet vibration patterns for guidance","authors":"Jérémy Plouzeau, Aida Erfanian, Cynthia Chiu, F. Mérienne, Yaoping Hu","doi":"10.1109/3DUI.2016.7460071","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460071","url":null,"abstract":"In this study, we present a preliminary exploration about the added value of vibration information for guiding navigation in a VE. The exploration consists of two parts. Firstly, we designed two different vibration patterns. These patterns, pushing pattern and compass pattern, differ conceptually in the levels of abstraction. Secondly, we undertook an experiment to compare the two patterns in guiding navigation in a VE. The objective of the comparison is to establish a baseline for examining the suitability of using vibration patterns to guide navigation.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115334952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2016-03-19DOI: 10.1109/3DUI.2016.7460046
B. Nuernberger, Kuo-Chin Lien, Tobias Höllerer, M. Turk
A 2D gesture annotation provides a simple way to annotate the physical world in augmented reality for a range of applications such as remote collaboration. When rendered from novel viewpoints, these annotations have previously only worked with statically positioned cameras or planar scenes. However, if the camera moves and is observing an arbitrary environment, 2D gesture annotations can easily lose their meaning when shown from novel viewpoints due to perspective effects. In this paper, we present a new approach towards solving this problem by using a gesture enhanced annotation interpretation. By first classifying which type of gesture the user drew, we show that it is possible to render the 2D annotations in 3D in a way that conforms more to the original intention of the user than with traditional methods. We first determined a generic vocabulary of important 2D gestures for an augmented reality enhanced remote collaboration scenario by running an Amazon Mechanical Turk study with 88 participants. Next, we designed a novel real-time method to automatically handle the two most common 2D gesture annotations - arrows and circles - and give a detailed analysis of the ambiguities that must be handled in each case. Arrow gestures are interpreted by identifying their anchor points and using scene surface normals for better perspective rendering. For circle gestures, we designed a novel energy function to help infer the object of interest using both 2D image cues and 3D geometric cues. Results indicate that our method outperforms previous approaches by better conveying the meaning of the original drawing from different viewpoints.
{"title":"Interpreting 2D gesture annotations in 3D augmented reality","authors":"B. Nuernberger, Kuo-Chin Lien, Tobias Höllerer, M. Turk","doi":"10.1109/3DUI.2016.7460046","DOIUrl":"https://doi.org/10.1109/3DUI.2016.7460046","url":null,"abstract":"A 2D gesture annotation provides a simple way to annotate the physical world in augmented reality for a range of applications such as remote collaboration. When rendered from novel viewpoints, these annotations have previously only worked with statically positioned cameras or planar scenes. However, if the camera moves and is observing an arbitrary environment, 2D gesture annotations can easily lose their meaning when shown from novel viewpoints due to perspective effects. In this paper, we present a new approach towards solving this problem by using a gesture enhanced annotation interpretation. By first classifying which type of gesture the user drew, we show that it is possible to render the 2D annotations in 3D in a way that conforms more to the original intention of the user than with traditional methods. We first determined a generic vocabulary of important 2D gestures for an augmented reality enhanced remote collaboration scenario by running an Amazon Mechanical Turk study with 88 participants. Next, we designed a novel real-time method to automatically handle the two most common 2D gesture annotations - arrows and circles - and give a detailed analysis of the ambiguities that must be handled in each case. Arrow gestures are interpreted by identifying their anchor points and using scene surface normals for better perspective rendering. For circle gestures, we designed a novel energy function to help infer the object of interest using both 2D image cues and 3D geometric cues. Results indicate that our method outperforms previous approaches by better conveying the meaning of the original drawing from different viewpoints.","PeriodicalId":175060,"journal":{"name":"2016 IEEE Symposium on 3D User Interfaces (3DUI)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124970962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：