首页 > 最新文献

2015 IEEE International Symposium on Mixed and Augmented Reality最新文献

英文 中文
[POSTER] Retrieving Lights Positions Using Plane Segmentation with Diffuse Illumination Reinforced with Specular Component [海报]使用平面分割与漫射照明增强与镜面组件检索光源位置
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.65
Paul-Emile Buteau, H. Saito
We present a novel method to retrieve multiple positions of point lights in real indoor scenes based on a 3D reconstruction. This method takes advantage of illumination over planes detected using a segmentation of the reconstructed mesh of the scene. We can also provide an estimation without suffering from the presence of specular highlights but rather use this component to refine the final estimation. This allows consistent relighting throughout the entire scene for aumented reality purposes.
提出了一种基于三维重建的室内真实场景点光源多位置检索方法。该方法利用场景重构网格分割检测到的平面上的光照。我们也可以提供一个不受镜面高光影响的估计,而是使用这个组件来改进最终的估计。这允许在整个场景中一致的重新照明,以达到增强现实的目的。
{"title":"[POSTER] Retrieving Lights Positions Using Plane Segmentation with Diffuse Illumination Reinforced with Specular Component","authors":"Paul-Emile Buteau, H. Saito","doi":"10.1109/ISMAR.2015.65","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.65","url":null,"abstract":"We present a novel method to retrieve multiple positions of point lights in real indoor scenes based on a 3D reconstruction. This method takes advantage of illumination over planes detected using a segmentation of the reconstructed mesh of the scene. We can also provide an estimation without suffering from the presence of specular highlights but rather use this component to refine the final estimation. This allows consistent relighting throughout the entire scene for aumented reality purposes.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125863673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
The Ventriloquist Effect in Augmented Reality 增强现实中的腹语者效应
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.18
Mikko Kytö, Kenta Kusumoto, P. Oittinen
An effective interaction in augmented reality (AR) requires utilization of different modalities. In this study, we investigated orienting the user in bimodal AR. Using auditory perception to support visual perception provides a useful approach for orienting the user to directions that are outside of the visual field-of-view (FOV). In particular, this is important in path-finding, where points-of-interest (POIs) can be all around the user. However, the ability to perceive the audio POIs is affected by the ventriloquism effect (VE), which means that audio POIs are captured by visual POIs. We measured the spatial limits for the VE in AR using a video see-through head-worn display. The results showed that the amount of the VE in AR was approx. 5°–15° higher than in a real environment. In AR, spatial disparity between an audio and visual POI should be at least 30° of azimuth angle, in order to perceive the audio and visual POIs as separate. The limit was affected by azimuth angle of visual POI and magnitude of head rotations. These results provide guidelines for designing bimodal AR systems.
增强现实(AR)中的有效交互需要利用不同的模式。在这项研究中,我们研究了在双峰AR中定位用户。使用听觉感知来支持视觉感知提供了一种有用的方法,可以将用户定向到视野(FOV)之外的方向。这在寻径中尤其重要,因为兴趣点(poi)可能就在用户周围。然而,感知音频点的能力受到腹语效应(VE)的影响,这意味着音频点被视觉点捕获。我们使用视频透明头戴式显示器测量了AR中VE的空间限制。结果表明,AR中VE的含量约为。5度;mdash; 15度;比真实环境要高。在AR中,音频和视觉POI之间的空间差异应该至少为30度;的方位角,以便感知音频和视觉点是分开的。该极限受视觉POI方位角和头部旋转幅度的影响。这些结果为设计双峰AR系统提供了指导。
{"title":"The Ventriloquist Effect in Augmented Reality","authors":"Mikko Kytö, Kenta Kusumoto, P. Oittinen","doi":"10.1109/ISMAR.2015.18","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.18","url":null,"abstract":"An effective interaction in augmented reality (AR) requires utilization of different modalities. In this study, we investigated orienting the user in bimodal AR. Using auditory perception to support visual perception provides a useful approach for orienting the user to directions that are outside of the visual field-of-view (FOV). In particular, this is important in path-finding, where points-of-interest (POIs) can be all around the user. However, the ability to perceive the audio POIs is affected by the ventriloquism effect (VE), which means that audio POIs are captured by visual POIs. We measured the spatial limits for the VE in AR using a video see-through head-worn display. The results showed that the amount of the VE in AR was approx. 5°–15° higher than in a real environment. In AR, spatial disparity between an audio and visual POI should be at least 30° of azimuth angle, in order to perceive the audio and visual POIs as separate. The limit was affected by azimuth angle of visual POI and magnitude of head rotations. These results provide guidelines for designing bimodal AR systems.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125200168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
[POSTER] Towards Estimating Usability Ratings of Handheld Augmented Reality Using Accelerometer Data [海报]使用加速计数据估计手持增强现实的可用性评级
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.62
M. Santos, Takafumi Taketomi, Goshiro Yamamoto, G. Klinker, C. Sandor, H. Kato
Usability evaluations are important to the development of augmented reality systems. However, conducting large-scale longitudinal studies remains challenging because of the lack of inexpensive but appropriate methods. In response, we propose a method for implicitly estimating usability ratings based on readily available sensor logs. To demonstrate our idea, we explored the use of features of accelerometer data in estimating usability ratings in an annotation task. Results show that our implicit method corresponds with explicit usability ratings at 79% and 84%. These results should be investigated further in other use cases, with other sensor logs.
可用性评估对增强现实系统的开发非常重要。然而,由于缺乏廉价但适当的方法,进行大规模的纵向研究仍然具有挑战性。作为回应,我们提出了一种基于现成的传感器日志隐式估计可用性评级的方法。为了演示我们的想法,我们探索了在注释任务中使用加速度计数据的特征来估计可用性评级。结果表明,我们的隐式方法对应于79%和84%的显式可用性评级。这些结果应该在其他用例中与其他传感器日志一起进一步研究。
{"title":"[POSTER] Towards Estimating Usability Ratings of Handheld Augmented Reality Using Accelerometer Data","authors":"M. Santos, Takafumi Taketomi, Goshiro Yamamoto, G. Klinker, C. Sandor, H. Kato","doi":"10.1109/ISMAR.2015.62","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.62","url":null,"abstract":"Usability evaluations are important to the development of augmented reality systems. However, conducting large-scale longitudinal studies remains challenging because of the lack of inexpensive but appropriate methods. In response, we propose a method for implicitly estimating usability ratings based on readily available sensor logs. To demonstrate our idea, we explored the use of features of accelerometer data in estimating usability ratings in an annotation task. Results show that our implicit method corresponds with explicit usability ratings at 79% and 84%. These results should be investigated further in other use cases, with other sensor logs.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122290842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
[POSTER] INCAST: Interactive Camera Streams for Surveillance Cams AR [海报]INCAST:交互式摄像头流监控摄像头AR
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.26
Istvan Szentandrasi, Michael Zachariá, Rudolf Kajan, J. Tinka, Markéta Dubská, Jakub Sochor, A. Herout
Augmented reality does not make any sense for fixed cameras. Or does it? In this work, we are dealing with static cameras and their usability for interactive augmented reality applications. Knowing that the camera does not move makes camera pose estimation both less and more difficult - one does not have to deal with pose change in time, but on the other hand, obtaining some level of understanding of the scene from a single viewpoint is challenging. We propose several ways how to gain advantage from the camera being static and a pipeline of a system for broadcasting a video stream enriched by information needed for its interactive visual augmenting - Interactive Camera Streams, INCAST. We present a proof-of-concept system showing the usability of INCAST on several use-cases - non-interactive demos and simple AR games.
增强现实对于固定摄像头来说没有任何意义。是吗?在这项工作中,我们正在处理静态相机及其在交互式增强现实应用中的可用性。知道相机不移动使得相机姿态估计既少又难——一个人不必处理姿势随时间的变化,但另一方面,从单一视点获得对场景的某种程度的理解是具有挑战性的。我们提出了几种方法,如何从静态摄像机和系统的管道中获得优势,以广播由交互式视觉增强所需的信息丰富的视频流-交互式摄像机流,INCAST。我们提出了一个概念验证系统,展示了INCAST在几个用例中的可用性——非交互式演示和简单的AR游戏。
{"title":"[POSTER] INCAST: Interactive Camera Streams for Surveillance Cams AR","authors":"Istvan Szentandrasi, Michael Zachariá, Rudolf Kajan, J. Tinka, Markéta Dubská, Jakub Sochor, A. Herout","doi":"10.1109/ISMAR.2015.26","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.26","url":null,"abstract":"Augmented reality does not make any sense for fixed cameras. Or does it? In this work, we are dealing with static cameras and their usability for interactive augmented reality applications. Knowing that the camera does not move makes camera pose estimation both less and more difficult - one does not have to deal with pose change in time, but on the other hand, obtaining some level of understanding of the scene from a single viewpoint is challenging. We propose several ways how to gain advantage from the camera being static and a pipeline of a system for broadcasting a video stream enriched by information needed for its interactive visual augmenting - Interactive Camera Streams, INCAST. We present a proof-of-concept system showing the usability of INCAST on several use-cases - non-interactive demos and simple AR games.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130638111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
[POSTER] Movable Spatial AR On-The-Go [海报]移动空间AR
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.55
Ahyun Lee, Joo-Haeng Lee, Jaehong Kim
We present a movable spatial augmented reality (SAR) system that can be easily installed in a user workspace. The proposed system aims to dynamically cover a wider projection area using a portable projector attached to a simple robotic device. It has a clear advantage than a conventional SAR scenario where, for example, a projector should be installe1d with a fixed projection area in the workspace. In the previous research [1], we proposed a data-driven kinematic control method for a movable SAR system. This method targets a SAR system integrated with a user-created robotic (UCR) device where an explicit kinematic configuration such as CAD model is unavailable. Our contribution in this paper is to show the feasibility of the data-driven control method by developing a practical application where dynamic change of projection area matters. We outline the control method and demonstrate an assembly guide example using a casually installed movable SAR system.
我们提出了一个可移动的空间增强现实(SAR)系统,可以很容易地安装在用户工作空间。提出的系统旨在动态覆盖更广泛的投影区域,使用便携式投影仪连接到一个简单的机器人设备。与传统的SAR场景相比,它有一个明显的优势,例如,投影仪应该在工作空间中安装一个固定的投影区域。在之前的研究[1]中,我们提出了一种数据驱动的移动SAR系统运动学控制方法。该方法的目标是SAR系统集成了用户创建的机器人(UCR)设备,其中没有明确的运动学配置,如CAD模型。我们在本文中的贡献是通过开发投影区域动态变化的实际应用来展示数据驱动控制方法的可行性。我们概述了控制方法,并演示了一个随机安装的可移动SAR系统的装配指南示例。
{"title":"[POSTER] Movable Spatial AR On-The-Go","authors":"Ahyun Lee, Joo-Haeng Lee, Jaehong Kim","doi":"10.1109/ISMAR.2015.55","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.55","url":null,"abstract":"We present a movable spatial augmented reality (SAR) system that can be easily installed in a user workspace. The proposed system aims to dynamically cover a wider projection area using a portable projector attached to a simple robotic device. It has a clear advantage than a conventional SAR scenario where, for example, a projector should be installe1d with a fixed projection area in the workspace. In the previous research [1], we proposed a data-driven kinematic control method for a movable SAR system. This method targets a SAR system integrated with a user-created robotic (UCR) device where an explicit kinematic configuration such as CAD model is unavailable. Our contribution in this paper is to show the feasibility of the data-driven control method by developing a practical application where dynamic change of projection area matters. We outline the control method and demonstrate an assembly guide example using a casually installed movable SAR system.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130909805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
[POSTER] ARPML: The Augmented Reality Process Modeling Language ARPML:增强现实过程建模语言
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.46
T. Muller, Tim Rieger
The successful application of augmented reality as a guidance tool for procedural tasks like maintenance or repair requires an easily usable way of modeling support processes. Even though some suggestions have already been made to address this problem, they still have shortcomings and don't provide all the required features. Thus in a first step the requirements a possible solution has to meet are collected and presented. Based on these, the augmented reality process modeling language (ARPML) is developed, which consists of the four building blocks (i) templates, (ii) sensors, (iii) work steps and (iv) tasks. In contrast to existing approaches it facilitates the creation of multiple views on a single process. This makes it possible to specifically select instructions and information needed in targeted work contexts. It also allows to combine multiple variants of one process into one model with only a minimum of redundancy. The application of ARPML is shown with a practical example.
增强现实作为程序性任务(如维护或修理)的指导工具的成功应用需要一种易于使用的建模支持过程的方法。尽管已经提出了一些建议来解决这个问题,但它们仍然有缺点,并且不能提供所有必需的功能。因此,在第一步中,收集并呈现可能的解决方案必须满足的需求。在此基础上,开发了增强现实过程建模语言(ARPML),该语言由四个构建块(i)模板,(ii)传感器,(iii)工作步骤和(iv)任务组成。与现有方法相比,它有助于在单个流程上创建多个视图。这使得在目标工作环境中具体选择所需的指令和信息成为可能。它还允许将一个流程的多个变体组合到一个模型中,只有最小的冗余。通过一个实例说明了ARPML的应用。
{"title":"[POSTER] ARPML: The Augmented Reality Process Modeling Language","authors":"T. Muller, Tim Rieger","doi":"10.1109/ISMAR.2015.46","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.46","url":null,"abstract":"The successful application of augmented reality as a guidance tool for procedural tasks like maintenance or repair requires an easily usable way of modeling support processes. Even though some suggestions have already been made to address this problem, they still have shortcomings and don't provide all the required features. Thus in a first step the requirements a possible solution has to meet are collected and presented. Based on these, the augmented reality process modeling language (ARPML) is developed, which consists of the four building blocks (i) templates, (ii) sensors, (iii) work steps and (iv) tasks. In contrast to existing approaches it facilitates the creation of multiple views on a single process. This makes it possible to specifically select instructions and information needed in targeted work contexts. It also allows to combine multiple variants of one process into one model with only a minimum of redundancy. The application of ARPML is shown with a practical example.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129883434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
[POSTER] Haptic Ring Interface Enabling Air-Writing in Virtual Reality Environment [海报]触觉环接口在虚拟现实环境中实现空中书写
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.37
Ki-Won Yeom, J. R. Kwon, Ju-Hyuck Maeng, Bum-Jae You
We introduce a novel finger worn ring interface that enables complex spatial interactions through 3D hand movement in virtual reality environment. Users receive physical feedback in the form of vibrations from the wearable ring interface as their finger reaches a certain 3D position. The positions of the fingertip are extracted, linked, and then reconstructed as a trajectory. This system allows the wearer to write characters in midair as if they were using an imaginary whiteboard. User can freely write in the air using Korean characters, English letters, both upper and lower case, and digits in real time with over 92% accuracy rate. Thus, it is now conceivable that anything people can do on contemporary touch based devices, they could do in midair with a pseudocontact interface.
我们介绍了一种新型的手指戴环界面,通过三维手部运动在虚拟现实环境中实现复杂的空间交互。当用户的手指到达某个3D位置时,可穿戴环界面会以振动的形式向用户提供物理反馈。指尖的位置被提取,链接,然后重建为一个轨迹。该系统允许佩戴者在半空中书写字符,就像他们在使用想象中的白板一样。用户可以自由地在空中实时书写韩文、英文字母、大小写字母、数字,准确率超过92%。因此,现在可以想象,人们可以在当代基于触摸的设备上做的任何事情,他们都可以在半空中用伪接触界面完成。
{"title":"[POSTER] Haptic Ring Interface Enabling Air-Writing in Virtual Reality Environment","authors":"Ki-Won Yeom, J. R. Kwon, Ju-Hyuck Maeng, Bum-Jae You","doi":"10.1109/ISMAR.2015.37","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.37","url":null,"abstract":"We introduce a novel finger worn ring interface that enables complex spatial interactions through 3D hand movement in virtual reality environment. Users receive physical feedback in the form of vibrations from the wearable ring interface as their finger reaches a certain 3D position. The positions of the fingertip are extracted, linked, and then reconstructed as a trajectory. This system allows the wearer to write characters in midair as if they were using an imaginary whiteboard. User can freely write in the air using Korean characters, English letters, both upper and lower case, and digits in real time with over 92% accuracy rate. Thus, it is now conceivable that anything people can do on contemporary touch based devices, they could do in midair with a pseudocontact interface.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116216989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
[POSTER] Realtime Shape-from-Template: System and Applications [海报]实时形状模板:系统和应用
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.35
T. Collins, A. Bartoli
An important yet unsolved problem in computer vision and Augmented Reality (AR) is to compute the 3D shape of nonrigid objects from live 2D videos. When the object's shape is provided in a rest pose, this is the Shape-from-Template (SfT) problem. Previous realtime SfT methods require simple, smooth templates, such as flat sheets of paper that are densely textured, and which deform in simple, smooth ways. We present a realtime SfT framework that handles generic template meshes, complex deformations and most of the difficulties present in real imaging conditions. Achieving this has required new, fast solutions to the two core sub-problems: robust registration and 3D shape inference. Registration is achieved with what we call Deformable Render-based Block Matching (DRBM): a highly-parallel solution which densely matches a time-varying render of the object to each video frame. We then combine matches from DRBM with physical deformation priors and perform shape inference, which is done by quickly solving a sparse linear system with a Geometric Multi-Grid (GMG)-based method. On a standard PC we achieve up to 21fps depending on the object. Source code will be released.
从实时二维视频中计算非刚性物体的三维形状是计算机视觉和增强现实(AR)领域尚未解决的一个重要问题。当物体的形状以静止姿态提供时,这就是模板形状(SfT)问题。以前的实时SfT方法需要简单、光滑的模板,比如纹理密集的平面纸张,它们以简单、光滑的方式变形。我们提出了一个实时SfT框架,处理通用模板网格,复杂变形和实际成像条件下存在的大多数困难。实现这一目标需要新的、快速的解决方案来解决两个核心子问题:鲁棒配准和3D形状推断。注册是通过我们称之为可变形的基于渲染的块匹配(DRBM)实现的:这是一种高度并行的解决方案,它将对象的时变渲染密集地匹配到每个视频帧。然后,我们将DRBM匹配与物理变形先验相结合,并通过基于几何多网格(Geometric Multi-Grid, GMG)的方法快速求解稀疏线性系统来进行形状推断。在标准PC上,根据对象的不同,我们可以达到21fps。源代码将被发布。
{"title":"[POSTER] Realtime Shape-from-Template: System and Applications","authors":"T. Collins, A. Bartoli","doi":"10.1109/ISMAR.2015.35","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.35","url":null,"abstract":"An important yet unsolved problem in computer vision and Augmented Reality (AR) is to compute the 3D shape of nonrigid objects from live 2D videos. When the object's shape is provided in a rest pose, this is the Shape-from-Template (SfT) problem. Previous realtime SfT methods require simple, smooth templates, such as flat sheets of paper that are densely textured, and which deform in simple, smooth ways. We present a realtime SfT framework that handles generic template meshes, complex deformations and most of the difficulties present in real imaging conditions. Achieving this has required new, fast solutions to the two core sub-problems: robust registration and 3D shape inference. Registration is achieved with what we call Deformable Render-based Block Matching (DRBM): a highly-parallel solution which densely matches a time-varying render of the object to each video frame. We then combine matches from DRBM with physical deformation priors and perform shape inference, which is done by quickly solving a sparse linear system with a Geometric Multi-Grid (GMG)-based method. On a standard PC we achieve up to 21fps depending on the object. Source code will be released.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114580040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
[POSTER] Augmenting Mobile C-arm Fluoroscopes via Stereo-RGBD Sensors for Multimodal Visualization [海报]基于立体rgbd传感器的增强移动c臂荧光显微镜的多模态可视化
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.24
S. Habert, Meng Ma, Wadim Kehl, X. Wang, Federico Tombari, P. Fallavollita, N. Navab
Fusing intraoperative X-ray data with real-time video in a common reference frame is not trivial since both modalities have to be acquired from the same viewpoint. The goal of this work is to design a flexible system comprising two RGBD sensors that can be attached to any mobile C-arm, with the objective of synthesizing projective color images from the X-ray source viewpoint. To achieve this, we calibrate the RGBD sensors to the X-ray source with a 3D calibration object. Then, we synthesize the projective color image from the X-ray viewpoint by applying a volumetric-based rendering method. Finally, the X-ray image is overlaid on the projective image without any further registration, offering a multimodal visualization of X-ray and color images. In this paper we present the different steps of development (i.e. hardware setup, calibration and rendering algorithm) and discuss clinical applications for the new video augmented C-arm. By placing X-ray markers on a hand patient and a spine model, we show that the overlay accuracy between the X-ray image and the synthetized image is in average 1.7 mm.
融合术中x线数据和实时视频在一个共同的参考框架是很重要的,因为这两种模式必须从同一视点获得。这项工作的目标是设计一个灵活的系统,包括两个RGBD传感器,可以附着在任何移动c臂上,目的是从x射线源的角度合成投影彩色图像。为了实现这一点,我们使用3D校准对象将RGBD传感器校准到x射线源。然后,采用基于体积的绘制方法合成x射线视点的投影彩色图像。最后,将x射线图像覆盖在投影图像上,无需进一步配准,从而提供x射线和彩色图像的多模态可视化。在本文中,我们介绍了不同的开发步骤(即硬件设置,校准和渲染算法),并讨论了新的视频增强c型臂的临床应用。通过在手部患者和脊柱模型上放置x射线标记,我们发现x射线图像与合成图像之间的覆盖精度平均为1.7 mm。
{"title":"[POSTER] Augmenting Mobile C-arm Fluoroscopes via Stereo-RGBD Sensors for Multimodal Visualization","authors":"S. Habert, Meng Ma, Wadim Kehl, X. Wang, Federico Tombari, P. Fallavollita, N. Navab","doi":"10.1109/ISMAR.2015.24","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.24","url":null,"abstract":"Fusing intraoperative X-ray data with real-time video in a common reference frame is not trivial since both modalities have to be acquired from the same viewpoint. The goal of this work is to design a flexible system comprising two RGBD sensors that can be attached to any mobile C-arm, with the objective of synthesizing projective color images from the X-ray source viewpoint. To achieve this, we calibrate the RGBD sensors to the X-ray source with a 3D calibration object. Then, we synthesize the projective color image from the X-ray viewpoint by applying a volumetric-based rendering method. Finally, the X-ray image is overlaid on the projective image without any further registration, offering a multimodal visualization of X-ray and color images. In this paper we present the different steps of development (i.e. hardware setup, calibration and rendering algorithm) and discuss clinical applications for the new video augmented C-arm. By placing X-ray markers on a hand patient and a spine model, we show that the overlay accuracy between the X-ray image and the synthetized image is in average 1.7 mm.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128277579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
[POSTER] Exploiting Photogrammetric Targets for Industrial AR [海报]利用工业AR的摄影测量目标
Pub Date : 2015-09-29 DOI: 10.1109/ISMAR.2015.42
Hemal Naik, Y. Oyamada, P. Keitler, Nassir Navab
In this work, we encourage the idea of using Photogrammetric targets for object tracking in Industrial Augmented Reality (IAR). Photogrammetric targets, especially uncoded circular targets, are widely used in the industry to perform 3D surface measurements. Therefore, an AR solution based on the uncoded circular targets can improve the work flow integration by reusing existing targets and saving time. These circular targets do not have coded patterns to establish unique 2D-3D correspondences between the targets on the model and their image projections. We solve this particular problem of 2D-3D correspondence of non-coplanar circular targets from a single image. We introduce a Conic pair descriptor, which computes the Eucledian invariants from circular targets in the model space and in the image space. A three stage method is used to compare the descriptors and compute the correspondences with up to 100% precision and 89% recall rates. We are able to achieve tracking performance of 3 FPS (2560x1920 pix) to 8 FPS (640×480 pix) depending on the camera resolution and the targets present in the scene.
在这项工作中,我们鼓励在工业增强现实(IAR)中使用摄影测量目标进行对象跟踪的想法。摄影测量目标,特别是未编码的圆形目标,在工业上广泛应用于三维表面测量。因此,基于未编码圆形目标的AR解决方案可以通过重用现有目标和节省时间来改善工作流集成。这些圆形目标没有编码模式,无法在模型上的目标与其图像投影之间建立独特的2D-3D对应关系。我们从单幅图像中解决了非共面圆形目标的2D-3D对应问题。我们引入了一个圆锥对描述子,它从模型空间和图像空间的圆形目标计算欧几里德不变量。采用三阶段方法对描述符进行比较并计算对应关系,准确率高达100%,召回率高达89%。我们能够实现3 FPS (2560x1920像素)到8 FPS (640×480像素)的跟踪性能,具体取决于相机分辨率和场景中存在的目标。
{"title":"[POSTER] Exploiting Photogrammetric Targets for Industrial AR","authors":"Hemal Naik, Y. Oyamada, P. Keitler, Nassir Navab","doi":"10.1109/ISMAR.2015.42","DOIUrl":"https://doi.org/10.1109/ISMAR.2015.42","url":null,"abstract":"In this work, we encourage the idea of using Photogrammetric targets for object tracking in Industrial Augmented Reality (IAR). Photogrammetric targets, especially uncoded circular targets, are widely used in the industry to perform 3D surface measurements. Therefore, an AR solution based on the uncoded circular targets can improve the work flow integration by reusing existing targets and saving time. These circular targets do not have coded patterns to establish unique 2D-3D correspondences between the targets on the model and their image projections. We solve this particular problem of 2D-3D correspondence of non-coplanar circular targets from a single image. We introduce a Conic pair descriptor, which computes the Eucledian invariants from circular targets in the model space and in the image space. A three stage method is used to compare the descriptors and compute the correspondences with up to 100% precision and 89% recall rates. We are able to achieve tracking performance of 3 FPS (2560x1920 pix) to 8 FPS (640×480 pix) depending on the camera resolution and the targets present in the scene.","PeriodicalId":240196,"journal":{"name":"2015 IEEE International Symposium on Mixed and Augmented Reality","volume":"130 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133912544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
2015 IEEE International Symposium on Mixed and Augmented Reality
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1