首页 > 最新文献

2011 10th IEEE International Symposium on Mixed and Augmented Reality最新文献

英文 中文
Information-theoretic database building and querying for mobile augmented reality applications 面向移动增强现实应用的信息论数据库构建与查询
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092369
P. Baheti, A. Swaminathan, Murali Chari, S. Diaz, Slawek Grzechnik
Recently, there has been tremendous interest in the area of mobile Augmented Reality (AR) with applications including navigation, social networking, gaming and education. Current generation mobile phones are equipped with camera, GPS and other sensors, e.g., magnetic compass, accelerometer, gyro in addition to having ever increasing computing/graphics capabilities and memory storage. Mobile AR applications process the output of one or more sensors to augment the real world view with useful information. This paper's focus is on the camera sensor output, and describes the building blocks for a vision-based AR system. We present information-theoretic techniques to build and maintain an image (feature) database based on reference images, and for querying the captured input images against this database. Performance results using standard image sets are provided demonstrating superior recognition performance even with dramatic reductions in feature database size.
最近,人们对移动增强现实(AR)领域产生了极大的兴趣,其应用包括导航、社交网络、游戏和教育。当前一代的手机除了具有不断增加的计算/图形能力和内存存储之外,还配备了摄像头,GPS和其他传感器,例如磁罗盘,加速度计,陀螺仪。移动增强现实应用程序处理一个或多个传感器的输出,用有用的信息增强真实世界的视野。本文的重点是相机传感器输出,并描述了基于视觉的AR系统的构建模块。我们提出了信息理论技术来建立和维护基于参考图像的图像(特征)数据库,并根据该数据库查询捕获的输入图像。使用标准图像集的性能结果显示,即使在特征数据库大小急剧减少的情况下,也具有优越的识别性能。
{"title":"Information-theoretic database building and querying for mobile augmented reality applications","authors":"P. Baheti, A. Swaminathan, Murali Chari, S. Diaz, Slawek Grzechnik","doi":"10.1109/ISMAR.2011.6092369","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092369","url":null,"abstract":"Recently, there has been tremendous interest in the area of mobile Augmented Reality (AR) with applications including navigation, social networking, gaming and education. Current generation mobile phones are equipped with camera, GPS and other sensors, e.g., magnetic compass, accelerometer, gyro in addition to having ever increasing computing/graphics capabilities and memory storage. Mobile AR applications process the output of one or more sensors to augment the real world view with useful information. This paper's focus is on the camera sensor output, and describes the building blocks for a vision-based AR system. We present information-theoretic techniques to build and maintain an image (feature) database based on reference images, and for querying the captured input images against this database. Performance results using standard image sets are provided demonstrating superior recognition performance even with dramatic reductions in feature database size.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127115780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Adaptive camera-based color mapping for mixed-reality applications 混合现实应用中基于自适应摄像头的色彩映射
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092382
Martin Knecht, C. Traxler, W. Purgathofer, M. Wimmer
We present a novel adaptive color mapping method for virtual objects in mixed-reality environments. In several mixed-reality applications, added virtual objects should be visually indistinguishable from real objects. Recent mixed-reality methods use global-illumination algorithms to approach this goal. However, simulating the light distribution is not enough for visually plausible images. Since the observing camera has its very own transfer function from real-world radiance values to RGB colors, virtual objects look artificial just because their rendered colors do not match with those of the camera. Our approach combines an on-line camera characterization method with a heuristic to map colors of virtual objects to colors as they would be seen by the observing camera. Previous tone-mapping functions were not designed for use in mixed-reality systems and thus did not take the camera-specific behavior into account. In contrast, our method takes the camera into account and thus can also handle changes of its parameters during runtime. The results show that virtual objects look visually more plausible than by just applying tone-mapping operators.
提出了一种新的混合现实环境中虚拟物体的自适应颜色映射方法。在一些混合现实应用程序中,添加的虚拟对象应该在视觉上与真实对象无法区分。最近的混合现实方法使用全局照明算法来实现这一目标。然而,模拟光的分布是不够的视觉上可信的图像。由于观察相机有自己的从真实世界的亮度值到RGB颜色的传递函数,虚拟物体看起来是人造的,只是因为它们的渲染颜色与相机的颜色不匹配。我们的方法结合了在线相机表征方法和启发式方法,将虚拟物体的颜色映射为观察相机所看到的颜色。以前的色调映射功能不是为混合现实系统设计的,因此没有考虑到相机特定的行为。相比之下,我们的方法考虑了相机,因此也可以在运行时处理其参数的变化。结果表明,虚拟对象在视觉上看起来比仅仅使用色调映射算子更可信。
{"title":"Adaptive camera-based color mapping for mixed-reality applications","authors":"Martin Knecht, C. Traxler, W. Purgathofer, M. Wimmer","doi":"10.1109/ISMAR.2011.6092382","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092382","url":null,"abstract":"We present a novel adaptive color mapping method for virtual objects in mixed-reality environments. In several mixed-reality applications, added virtual objects should be visually indistinguishable from real objects. Recent mixed-reality methods use global-illumination algorithms to approach this goal. However, simulating the light distribution is not enough for visually plausible images. Since the observing camera has its very own transfer function from real-world radiance values to RGB colors, virtual objects look artificial just because their rendered colors do not match with those of the camera. Our approach combines an on-line camera characterization method with a heuristic to map colors of virtual objects to colors as they would be seen by the observing camera. Previous tone-mapping functions were not designed for use in mixed-reality systems and thus did not take the camera-specific behavior into account. In contrast, our method takes the camera into account and thus can also handle changes of its parameters during runtime. The results show that virtual objects look visually more plausible than by just applying tone-mapping operators.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114886367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 16
Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances 利用自中心视觉实现磁干扰下的鲁棒惯性体跟踪
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092528
G. Bleser, Gustaf Hendeby, M. Miezal
In the context of a smart user assistance system for industrial manipulation tasks it is necessary to capture motions of the upper body and limbs of the worker in order to derive his or her interactions with the task space. While such capturing technology already exists, the novelty of the proposed work results from the strong requirements of the application context: The method should be flexible and use only on-body sensors, work accurately in industrial environments that suffer from severe magnetic disturbances, and enable consistent registration between the user body frame and the task space. Currently available systems cannot provide this. This paper suggests a novel egocentric solution for visual-inertial upper-body motion tracking based on recursive filtering and model-based sensor fusion. Visual detections of the wrists in the images of a chest-mounted camera are used as substitute for the commonly used magnetometer measurements. The on-body sensor network, the motion capturing system, and the required calibration procedure are described and successful operation is shown in a real industrial environment.
在工业操作任务的智能用户辅助系统的背景下,有必要捕获工人的上半身和四肢的运动,以便派生他或她与任务空间的交互。虽然这种捕获技术已经存在,但所提出的工作的新颖性来自于应用环境的强烈要求:该方法应该是灵活的,只使用身体上的传感器,在遭受严重磁干扰的工业环境中准确工作,并在用户身体框架和任务空间之间实现一致的注册。当前可用的系统无法提供此功能。提出了一种基于递归滤波和基于模型的传感器融合的视觉惯性上体运动跟踪自中心算法。在安装在胸前的相机图像中对手腕的视觉检测被用来代替常用的磁力计测量。描述了人体传感器网络、运动捕捉系统和所需的校准程序,并在实际工业环境中成功运行。
{"title":"Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances","authors":"G. Bleser, Gustaf Hendeby, M. Miezal","doi":"10.1109/ISMAR.2011.6092528","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092528","url":null,"abstract":"In the context of a smart user assistance system for industrial manipulation tasks it is necessary to capture motions of the upper body and limbs of the worker in order to derive his or her interactions with the task space. While such capturing technology already exists, the novelty of the proposed work results from the strong requirements of the application context: The method should be flexible and use only on-body sensors, work accurately in industrial environments that suffer from severe magnetic disturbances, and enable consistent registration between the user body frame and the task space. Currently available systems cannot provide this. This paper suggests a novel egocentric solution for visual-inertial upper-body motion tracking based on recursive filtering and model-based sensor fusion. Visual detections of the wrists in the images of a chest-mounted camera are used as substitute for the commonly used magnetometer measurements. The on-body sensor network, the motion capturing system, and the required calibration procedure are described and successful operation is shown in a real industrial environment.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130495613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 40
Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras 无阻碍的远程呈现系统,实时3D捕获和显示使用商品深度相机
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092379
Andrew Maimone, H. Fuchs
This paper introduces a proof-of-concept telepresence system that offers fully dynamic, real-time 3D scene capture and continuous-viewpoint, head-tracked stereo 3D display without requiring the user to wear any tracking or viewing apparatus. We present a complete software and hardware framework for implementing the system, which is based on an array of commodity Microsoft Kinect™color-plus-depth cameras. Novel contributions include an algorithm for merging data between multiple depth cameras and techniques for automatic color calibration and preserving stereo quality even with low rendering rates. Also presented is a solution to the problem of interference that occurs between Kinect cameras with overlapping views. Emphasis is placed on a fully GPU-accelerated data processing and rendering pipeline that can apply hole filling, smoothing, data merger, surface generation, and color correction at rates of up to 100 million triangles/sec on a single PC and graphics board. Also presented is a Kinect-based marker-less tracking system that combines 2D eye recognition with depth information to allow head-tracked stereo views to be rendered for a parallax barrier autostereoscopic display. Our system is affordable and reproducible, offering the opportunity to easily deliver 3D telepresence beyond the researcher's lab.
本文介绍了一种概念验证远程呈现系统,该系统提供完全动态,实时3D场景捕获和连续视点,头部跟踪立体3D显示,而无需用户佩戴任何跟踪或观看设备。我们提出了一个完整的软件和硬件框架来实现系统,它是基于一系列商品微软Kinect™彩色加深度相机。新颖的贡献包括在多个深度相机之间合并数据的算法和自动颜色校准技术,即使在低渲染率下也能保持立体质量。此外,还提出了一种解决Kinect摄像头之间出现重叠视图的干扰问题的方法。重点放在完全gpu加速的数据处理和渲染管道上,可以在单个PC和图形板上以高达1亿个三角形/秒的速率应用孔填充,平滑,数据合并,表面生成和颜色校正。此外,还介绍了一种基于kinect的无标记跟踪系统,该系统将2D眼睛识别与深度信息相结合,允许为视差屏障自动立体显示呈现头部跟踪的立体视图。我们的系统价格合理且可复制,提供了在研究人员实验室之外轻松提供3D远程呈现的机会。
{"title":"Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras","authors":"Andrew Maimone, H. Fuchs","doi":"10.1109/ISMAR.2011.6092379","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092379","url":null,"abstract":"This paper introduces a proof-of-concept telepresence system that offers fully dynamic, real-time 3D scene capture and continuous-viewpoint, head-tracked stereo 3D display without requiring the user to wear any tracking or viewing apparatus. We present a complete software and hardware framework for implementing the system, which is based on an array of commodity Microsoft Kinect™color-plus-depth cameras. Novel contributions include an algorithm for merging data between multiple depth cameras and techniques for automatic color calibration and preserving stereo quality even with low rendering rates. Also presented is a solution to the problem of interference that occurs between Kinect cameras with overlapping views. Emphasis is placed on a fully GPU-accelerated data processing and rendering pipeline that can apply hole filling, smoothing, data merger, surface generation, and color correction at rates of up to 100 million triangles/sec on a single PC and graphics board. Also presented is a Kinect-based marker-less tracking system that combines 2D eye recognition with depth information to allow head-tracked stereo views to be rendered for a parallax barrier autostereoscopic display. Our system is affordable and reproducible, offering the opportunity to easily deliver 3D telepresence beyond the researcher's lab.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134391616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 239
Creating hybrid user interfaces with a 2D multi-touch tabletop and a 3D see-through head-worn display 使用2D多点触控桌面和3D透明头戴式显示器创建混合用户界面
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092391
N. Dedual, Ohan Oda, Steven K. Feiner
How can multiple different display and interaction devices be used together to create an effective augmented reality environment? We explore the design of several prototype hybrid user interfaces that combine a 2D multi-touch tabletop display with a 3D head-tracked video-see-through display. We describe a simple modeling application and an urban visualization tool in which the information presented on the head-worn display supplements the information displayed on the tabletop, using a variety of approaches to track the head-worn display relative to the tabletop. In all cases, our goal is to allow users who can see only the tabletop to interact effectively with users wearing head-worn displays.
如何使用多种不同的显示和交互设备来创建一个有效的增强现实环境?我们探索了几个原型混合用户界面的设计,将2D多点触摸桌面显示器与3D头显跟踪视频透明显示器相结合。我们描述了一个简单的建模应用程序和一个城市可视化工具,其中头戴式显示器上显示的信息补充了桌面显示的信息,使用各种方法来跟踪相对于桌面的头戴式显示器。在所有情况下,我们的目标是让只能看到桌面的用户与戴着头戴式显示器的用户进行有效的交互。
{"title":"Creating hybrid user interfaces with a 2D multi-touch tabletop and a 3D see-through head-worn display","authors":"N. Dedual, Ohan Oda, Steven K. Feiner","doi":"10.1109/ISMAR.2011.6092391","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092391","url":null,"abstract":"How can multiple different display and interaction devices be used together to create an effective augmented reality environment? We explore the design of several prototype hybrid user interfaces that combine a 2D multi-touch tabletop display with a 3D head-tracked video-see-through display. We describe a simple modeling application and an urban visualization tool in which the information presented on the head-worn display supplements the information displayed on the tabletop, using a variety of approaches to track the head-worn display relative to the tabletop. In all cases, our goal is to allow users who can see only the tabletop to interact effectively with users wearing head-worn displays.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134195749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
Building your vision with Qualcomm's Mobile Augmented Reality (AR) platform: AR on mobile devices 利用高通的移动增强现实(AR)平台构建您的愿景:移动设备上的AR
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092355
Daniel Wagner, I. Barakonyi, Istvan Siklossy, Jay Wright, R. Ashok, S. Diaz, B. MacIntyre, D. Schmalstieg
Academic/Research Overview of Mobile Augmented Reality Introduction to the Qualcomm AR Platform — Features and Usage Scenarios Developing Mobile AR Applications using Qualcomm's Unity Extension Introduction to Qualcomm's QCAR SDK Native API Cross Platform Development with the Native QCAR SDK
高通AR平台简介-使用高通Unity扩展开发移动AR应用程序的功能和使用场景介绍高通QCAR SDK原生API跨平台开发原生QCAR SDK
{"title":"Building your vision with Qualcomm's Mobile Augmented Reality (AR) platform: AR on mobile devices","authors":"Daniel Wagner, I. Barakonyi, Istvan Siklossy, Jay Wright, R. Ashok, S. Diaz, B. MacIntyre, D. Schmalstieg","doi":"10.1109/ISMAR.2011.6092355","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092355","url":null,"abstract":"Academic/Research Overview of Mobile Augmented Reality Introduction to the Qualcomm AR Platform — Features and Usage Scenarios Developing Mobile AR Applications using Qualcomm's Unity Extension Introduction to Qualcomm's QCAR SDK Native API Cross Platform Development with the Native QCAR SDK","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133033134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Light factorization for mixed-frequency shadows in augmented reality 增强现实中混合频率阴影的光分解
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092384
D. Nowrouzezahrai, S. Geiger, Kenny Mitchell, R. Sumner, Wojciech Jarosz, M. Gross
Integrating animated virtual objects with their surroundings for high-quality augmented reality requires both geometric and radio-metric consistency. We focus on the latter of these problems and present an approach that captures and factorizes external lighting in a manner that allows for realistic relighting of both animated and static virtual objects. Our factorization facilitates a combination of hard and soft shadows, with high-performance, in a manner that is consistent with the surrounding scene lighting.
为实现高质量的增强现实,将动画虚拟对象与其周围环境整合需要几何和辐射度量的一致性。我们专注于这些问题的后一个,并提出了一种方法,以一种允许动画和静态虚拟对象的现实重照明的方式捕获和分解外部照明。我们的分解促进了硬阴影和软阴影的结合,以一种与周围场景照明一致的方式,具有高性能。
{"title":"Light factorization for mixed-frequency shadows in augmented reality","authors":"D. Nowrouzezahrai, S. Geiger, Kenny Mitchell, R. Sumner, Wojciech Jarosz, M. Gross","doi":"10.1109/ISMAR.2011.6092384","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092384","url":null,"abstract":"Integrating animated virtual objects with their surroundings for high-quality augmented reality requires both geometric and radio-metric consistency. We focus on the latter of these problems and present an approach that captures and factorizes external lighting in a manner that allows for realistic relighting of both animated and static virtual objects. Our factorization facilitates a combination of hard and soft shadows, with high-performance, in a manner that is consistent with the surrounding scene lighting.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114143907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 47
RGB-D camera-based parallel tracking and meshing 基于RGB-D摄像机的并行跟踪和网格划分
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092380
S. Lieberknecht, Andrea Huber, Slobodan Ilic, Selim Benhimane
Compared to standard color cameras, RGB-D cameras are designed to additionally provide the depth of imaged pixels which in turn results in a dense colored 3D point cloud representing the environment from a certain viewpoint. We present a real-time tracking method that performs motion estimation of a consumer RGB-D camera with respect to an unknown environment while at the same time reconstructing this environment as a dense textured mesh. Unlike parallel tracking and mapping performed with a standard color or grey scale camera, tracking with an RGB-D camera allows a correctly scaled camera motion estimation. Therefore, there is no need for measuring the environment by any additional tool or equipping the environment by placing objects in it with known sizes. The tracking can be directly started and does not require any preliminary known and/or constrained camera motion. The colored point clouds obtained from every RGB-D image are used to create textured meshes representing the environment from a certain camera view and the real-time estimated camera motion is used to correctly align these meshes over time in order to combine them into a dense reconstruction of the environment. We quantitatively evaluated the proposed method using real image sequences of a challenging scenario and their corresponding ground truth motion obtained with a mechanical measurement arm. We also compared it to a commonly used state-of-the-art method where only the color information is used. We show the superiority of the proposed tracking in terms of accuracy, robustness and usability. We also demonstrate its usage in several Augmented Reality scenarios where the tracking allows a reliable camera motion estimation and the meshing increases the realism of the augmentations by correctly handling their occlusions.
与标准彩色相机相比,RGB-D相机的设计额外提供了图像像素的深度,这反过来又产生了一个密集的彩色3D点云,从某个角度代表环境。我们提出了一种实时跟踪方法,该方法对消费者RGB-D相机在未知环境中进行运动估计,同时将该环境重建为密集纹理网格。与使用标准彩色或灰度相机执行的并行跟踪和映射不同,使用RGB-D相机进行跟踪可以正确缩放相机运动估计。因此,不需要通过任何额外的工具来测量环境,也不需要通过在环境中放置已知尺寸的物体来装备环境。跟踪可以直接开始,不需要任何初步的已知和/或约束相机运动。从每个RGB-D图像中获得的彩色点云用于创建代表特定摄像机视图的环境的纹理网格,并使用实时估计的摄像机运动来随着时间的推移正确对齐这些网格,以便将它们组合成密集的环境重建。我们使用具有挑战性场景的真实图像序列及其对应的机械测量臂获得的地面真值运动来定量评估所提出的方法。我们还将其与仅使用颜色信息的常用最先进方法进行了比较。我们证明了所提出的跟踪在准确性、鲁棒性和可用性方面的优越性。我们还演示了它在几个增强现实场景中的使用,其中跟踪允许可靠的相机运动估计,网格通过正确处理它们的遮挡来增加增强的真实感。
{"title":"RGB-D camera-based parallel tracking and meshing","authors":"S. Lieberknecht, Andrea Huber, Slobodan Ilic, Selim Benhimane","doi":"10.1109/ISMAR.2011.6092380","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092380","url":null,"abstract":"Compared to standard color cameras, RGB-D cameras are designed to additionally provide the depth of imaged pixels which in turn results in a dense colored 3D point cloud representing the environment from a certain viewpoint. We present a real-time tracking method that performs motion estimation of a consumer RGB-D camera with respect to an unknown environment while at the same time reconstructing this environment as a dense textured mesh. Unlike parallel tracking and mapping performed with a standard color or grey scale camera, tracking with an RGB-D camera allows a correctly scaled camera motion estimation. Therefore, there is no need for measuring the environment by any additional tool or equipping the environment by placing objects in it with known sizes. The tracking can be directly started and does not require any preliminary known and/or constrained camera motion. The colored point clouds obtained from every RGB-D image are used to create textured meshes representing the environment from a certain camera view and the real-time estimated camera motion is used to correctly align these meshes over time in order to combine them into a dense reconstruction of the environment. We quantitatively evaluated the proposed method using real image sequences of a challenging scenario and their corresponding ground truth motion obtained with a mechanical measurement arm. We also compared it to a commonly used state-of-the-art method where only the color information is used. We show the superiority of the proposed tracking in terms of accuracy, robustness and usability. We also demonstrate its usage in several Augmented Reality scenarios where the tracking allows a reliable camera motion estimation and the meshing increases the realism of the augmentations by correctly handling their occlusions.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129972024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 42
Toward augmenting everything: Detecting and tracking geometrical features on planar objects 向增强一切:检测和跟踪平面物体的几何特征
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092366
Hideaki Uchiyama, É. Marchand
This paper presents an approach for detecting and tracking various types of planar objects with geometrical features. We combine traditional keypoint detectors with Locally Likely Arrangement Hashing (LLAH) [21] for geometrical feature based keypoint matching. Because the stability of keypoint extraction affects the accuracy of the keypoint matching, we set the criteria of keypoint selection on keypoint response and the distance between keypoints. In order to produce robustness to scale changes, we build a non-uniform image pyramid according to keypoint distribution at each scale. In the experiments, we evaluate the applicability of traditional keypoint detectors with LLAH for the detection. We also compare our approach with SURF and finally demonstrate that it is possible to detect and track different types of textures including colorful pictures, binary fiducial markers and handwritings.
本文提出了一种检测和跟踪具有几何特征的各类平面目标的方法。我们将传统的关键点检测器与局部似然排列哈希(LLAH)[21]相结合,用于基于几何特征的关键点匹配。由于关键点提取的稳定性影响到关键点匹配的准确性,我们将关键点的选择标准设定为关键点响应和关键点之间的距离。为了对尺度变化产生鲁棒性,我们根据每个尺度上的关键点分布构建了一个非均匀的图像金字塔。在实验中,我们评估了传统的LLAH关键点检测器在检测中的适用性。我们还将我们的方法与SURF进行了比较,并最终证明可以检测和跟踪不同类型的纹理,包括彩色图片,二进制基准标记和手写。
{"title":"Toward augmenting everything: Detecting and tracking geometrical features on planar objects","authors":"Hideaki Uchiyama, É. Marchand","doi":"10.1109/ISMAR.2011.6092366","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092366","url":null,"abstract":"This paper presents an approach for detecting and tracking various types of planar objects with geometrical features. We combine traditional keypoint detectors with Locally Likely Arrangement Hashing (LLAH) [21] for geometrical feature based keypoint matching. Because the stability of keypoint extraction affects the accuracy of the keypoint matching, we set the criteria of keypoint selection on keypoint response and the distance between keypoints. In order to produce robustness to scale changes, we build a non-uniform image pyramid according to keypoint distribution at each scale. In the experiments, we evaluate the applicability of traditional keypoint detectors with LLAH for the detection. We also compare our approach with SURF and finally demonstrate that it is possible to detect and track different types of textures including colorful pictures, binary fiducial markers and handwritings.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126409862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 20
A user study on the Snap-To-Feature interaction method 用户对Snap-To-Feature交互方法的研究
Pub Date : 2011-10-26 DOI: 10.1109/ISMAR.2011.6092398
Gun A. Lee, M. Billinghurst
Recent advances in mobile computing and augmented reality (AR) technology have lead to popularization of mobile AR applications. Touch screen input is common in mobile devices, and also widely used in mobile AR applications. However, due to unsteady camera view movement, it can be hard to carry out precise interactions in handheld AR environments, for tasks such as tracing physical objects. In this research, we investigate a Snap-To-Feature interaction method that helps users to perform more accurate touch screen interactions by attracting user input points to image features in the AR scene. A user experiment is performed using the method to trace a physical object, which is typical for modeling real objects within the AR scene. The results shows that the Snap-To-Feature method makes a significant difference in the accuracy of touch screen based AR interaction.
移动计算和增强现实(AR)技术的最新进展导致了移动AR应用的普及。触摸屏输入在移动设备中很常见,在移动AR应用中也有广泛的应用。然而,由于相机视角移动不稳定,在手持式AR环境中很难进行精确的交互,例如跟踪物理对象。在本研究中,我们研究了一种Snap-To-Feature交互方法,该方法通过吸引用户输入点到AR场景中的图像特征来帮助用户执行更准确的触摸屏交互。使用该方法进行用户实验以跟踪物理对象,这是AR场景中典型的真实对象建模。结果表明,Snap-To-Feature方法在基于触摸屏的AR交互精度方面有显著差异。
{"title":"A user study on the Snap-To-Feature interaction method","authors":"Gun A. Lee, M. Billinghurst","doi":"10.1109/ISMAR.2011.6092398","DOIUrl":"https://doi.org/10.1109/ISMAR.2011.6092398","url":null,"abstract":"Recent advances in mobile computing and augmented reality (AR) technology have lead to popularization of mobile AR applications. Touch screen input is common in mobile devices, and also widely used in mobile AR applications. However, due to unsteady camera view movement, it can be hard to carry out precise interactions in handheld AR environments, for tasks such as tracing physical objects. In this research, we investigate a Snap-To-Feature interaction method that helps users to perform more accurate touch screen interactions by attracting user input points to image features in the AR scene. A user experiment is performed using the method to trace a physical object, which is typical for modeling real objects within the AR scene. The results shows that the Snap-To-Feature method makes a significant difference in the accuracy of touch screen based AR interaction.","PeriodicalId":298757,"journal":{"name":"2011 10th IEEE International Symposium on Mixed and Augmented Reality","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126667073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
期刊
2011 10th IEEE International Symposium on Mixed and Augmented Reality
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1