Specularities, which are often visible in images, may be problematic in computer vision since they depend on parameters which are difficult to estimate in practice. We present an empirical model called JOLIMAS: JOint LIght-MAterial Specularity, which allows specularity prediction. JOLIMAS is reconstructed from images of specular reflections observed on a planar surface and implicitly includes light and material properties which are intrinsic to specularities. This work was motivated by the observation that specularities have a conic shape on planar surfaces. A theoretical study on the well known illumination models of Phong and Blinn-Phong was conducted to support the accuracy of this hypothesis. A conic shape is obtained by projecting a quadric on a planar surface. We showed empirically the existence of a fixed quadric whose perspective projection fits the conic shaped specularity in the associated image. JOLIMAS predicts the complex phenomenon of specularity using a simple geometric approach with static parameters on the object material and on the light source shape. It is adapted to indoor light sources such as light bulbs or fluorescent lamps. The performance of the prediction was convincing on synthetic and real sequences. Additionally, we used the specularity prediction for dynamic retexturing and obtained convincing rendering results. Further results are presented as supplementary material.
{"title":"An Empirical Model for Specularity Prediction with Application to Dynamic Retexturing","authors":"Alexandre Morgand, M. Tamaazousti, A. Bartoli","doi":"10.1109/ISMAR.2016.13","DOIUrl":"https://doi.org/10.1109/ISMAR.2016.13","url":null,"abstract":"Specularities, which are often visible in images, may be problematic in computer vision since they depend on parameters which are difficult to estimate in practice. We present an empirical model called JOLIMAS: JOint LIght-MAterial Specularity, which allows specularity prediction. JOLIMAS is reconstructed from images of specular reflections observed on a planar surface and implicitly includes light and material properties which are intrinsic to specularities. This work was motivated by the observation that specularities have a conic shape on planar surfaces. A theoretical study on the well known illumination models of Phong and Blinn-Phong was conducted to support the accuracy of this hypothesis. A conic shape is obtained by projecting a quadric on a planar surface. We showed empirically the existence of a fixed quadric whose perspective projection fits the conic shaped specularity in the associated image. JOLIMAS predicts the complex phenomenon of specularity using a simple geometric approach with static parameters on the object material and on the light source shape. It is adapted to indoor light sources such as light bulbs or fluorescent lamps. The performance of the prediction was convincing on synthetic and real sequences. Additionally, we used the specularity prediction for dynamic retexturing and obtained convincing rendering results. Further results are presented as supplementary material.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131586407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas Richter-Trummer, Denis Kalkofen, Jinwoo Park, D. Schmalstieg
We present a method for recovering both incident lighting and surface materials from casually scanned geometry. By casual, we mean a rapid and potentially noisy scanning procedure of unmodified and uninstrumented scenes with a commodity RGB-D sensor. In other words, unlike reconstruction procedures which require careful preparations in a laboratory environment, our method works with input that can be obtained by consumer users. To ensure a robust procedure, we segment the reconstructed geometry into surfaces with homogeneous material properties and compute the radiance transfer on these segments. With this input, we solve the inverse rendering problem of factorization into lighting and material properties using an iterative optimization in spherical harmonics form. This allows us to account for self-shadowing and recover specular properties. The resulting data can be used to generate a wide range of mixed reality applications, including the rendering of synthetic objects with matching lighting into a given scene, but also re-rendering the scene (or a part of it) with new lighting. We show the robustness of our approach with real and synthetic examples under a variety of lighting conditions and compare them with ground truth data.
{"title":"Instant Mixed Reality Lighting from Casual Scanning","authors":"Thomas Richter-Trummer, Denis Kalkofen, Jinwoo Park, D. Schmalstieg","doi":"10.1109/ISMAR.2016.18","DOIUrl":"https://doi.org/10.1109/ISMAR.2016.18","url":null,"abstract":"We present a method for recovering both incident lighting and surface materials from casually scanned geometry. By casual, we mean a rapid and potentially noisy scanning procedure of unmodified and uninstrumented scenes with a commodity RGB-D sensor. In other words, unlike reconstruction procedures which require careful preparations in a laboratory environment, our method works with input that can be obtained by consumer users. To ensure a robust procedure, we segment the reconstructed geometry into surfaces with homogeneous material properties and compute the radiance transfer on these segments. With this input, we solve the inverse rendering problem of factorization into lighting and material properties using an iterative optimization in spherical harmonics form. This allows us to account for self-shadowing and recover specular properties. The resulting data can be used to generate a wide range of mixed reality applications, including the rendering of synthetic objects with matching lighting into a given scene, but also re-rendering the scene (or a part of it) with new lighting. We show the robustness of our approach with real and synthetic examples under a variety of lighting conditions and compare them with ground truth data.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122496215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Proper scene inference provides the basis for a seamless integration of virtual objects into the real environment. While widely neglected in many AR/MR environments, previous approaches providing good results were based on rather complex setups, often involving mirrored balls, several HDR cameras, and fish eye lenses to achieve proper light probes. In this paper we present an approach requiring a single RGB-D camera image only for generating glossy reflections on virtual objects. Our approach is based on a partial 3D reconstruction of the real environment combined with a screen-space ray-tracing mechanism. We show that our approach allows for convincing reflections of the real environment as well as mutual reflections between virtual objects of an MR environment.
{"title":"A Single Camera Image Based Approach for Glossy Reflections in Mixed Reality Applications","authors":"Tobias Schwandt, W. Broll","doi":"10.1109/ISMAR.2016.12","DOIUrl":"https://doi.org/10.1109/ISMAR.2016.12","url":null,"abstract":"Proper scene inference provides the basis for a seamless integration of virtual objects into the real environment. While widely neglected in many AR/MR environments, previous approaches providing good results were based on rather complex setups, often involving mirrored balls, several HDR cameras, and fish eye lenses to achieve proper light probes. In this paper we present an approach requiring a single RGB-D camera image only for generating glossy reflections on virtual objects. Our approach is based on a partial 3D reconstruction of the real environment combined with a screen-space ray-tracing mechanism. We show that our approach allows for convincing reflections of the real environment as well as mutual reflections between virtual objects of an MR environment.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132067774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present an approach to estimate absolute scale in handheld monocular SLAM by simultaneously tracking the user's face with a user-facing camera while a world-facing camera captures the scene for localization and mapping. Given face tracking at absolute scale, two images of a face taken from two different viewpoints enable estimating the translational distance between the two viewpoints in absolute units, such as millimeters. Under the assumption that the face itself stayed stationary in the scene while taking the two images, the motion of the user-facing camera relative to the face can be transferred to the motion of the rigidly connected world-facing camera relative to the scene. This allows determining also the latter motion in absolute units and enables reconstructing and tracking the scene at absolute scale.As faces of different adult humans differ only moderately in terms of size, it is possible to rely on statistics for guessing the absolute dimensions of a face. For improved accuracy the dimensions of the particular face of the user can be calibrated.Based on sequences of world-facing and user-facing images captured by a mobile phone, we show for different scenes how our approach enables reconstruction and tracking at absolute scale using a proof-of-concept implementation. Quantitative evaluations against ground truth data confirm that our approach provides absolute scale at an accuracy well suited for different applications. Particularly, we show how our method enables various use cases in handheld Augmented Reality applications that superimpose virtual objects at absolute scale or feature interactive distance measurements.
{"title":"Leveraging the User's Face for Absolute Scale Estimation in Handheld Monocular SLAM","authors":"S. Knorr, Daniel Kurz","doi":"10.1109/ISMAR.2016.20","DOIUrl":"https://doi.org/10.1109/ISMAR.2016.20","url":null,"abstract":"We present an approach to estimate absolute scale in handheld monocular SLAM by simultaneously tracking the user's face with a user-facing camera while a world-facing camera captures the scene for localization and mapping. Given face tracking at absolute scale, two images of a face taken from two different viewpoints enable estimating the translational distance between the two viewpoints in absolute units, such as millimeters. Under the assumption that the face itself stayed stationary in the scene while taking the two images, the motion of the user-facing camera relative to the face can be transferred to the motion of the rigidly connected world-facing camera relative to the scene. This allows determining also the latter motion in absolute units and enables reconstructing and tracking the scene at absolute scale.As faces of different adult humans differ only moderately in terms of size, it is possible to rely on statistics for guessing the absolute dimensions of a face. For improved accuracy the dimensions of the particular face of the user can be calibrated.Based on sequences of world-facing and user-facing images captured by a mobile phone, we show for different scenes how our approach enables reconstruction and tracking at absolute scale using a proof-of-concept implementation. Quantitative evaluations against ground truth data confirm that our approach provides absolute scale at an accuracy well suited for different applications. Particularly, we show how our method enables various use cases in handheld Augmented Reality applications that superimpose virtual objects at absolute scale or feature interactive distance measurements.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126838145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present TactileVR, a proof-of-concept virtual reality system in which a user is free to move around and interact with physical objects and toys, which are represented in the virtual world. By integrating tracking information from the head, hands and feet of the user, as well as the objects, we infer complex gestures and interactions such as shaking a toy, rotating a steering wheel, or clapping your hands. We create educational and recreational experiences for kids, which promote exploration and discovery, while feeling intuitive and safe. In each experience objects have a unique appearance and behavior e.g. in an electric circuits lab toy blocks serve as switches, batteries and light bulbs.We conducted a user study with children ages 5-11, who experienced TactileVR and interacted with virtual proxies of physical objects. Children took instantly to the TactileVR environment, intuitively discovered a variety of interactions, and completed tasks faster than with non-tactile virtual objects. Moreover, the presence of physical toys created the opportunity for collaborative play, even when only some of the kids were using a VR headset.
{"title":"TactileVR: Integrating Physical Toys into Learn and Play Virtual Reality Experiences","authors":"Lior Shapira, J. Amores, X. Benavides","doi":"10.1109/ISMAR.2016.25","DOIUrl":"https://doi.org/10.1109/ISMAR.2016.25","url":null,"abstract":"We present TactileVR, a proof-of-concept virtual reality system in which a user is free to move around and interact with physical objects and toys, which are represented in the virtual world. By integrating tracking information from the head, hands and feet of the user, as well as the objects, we infer complex gestures and interactions such as shaking a toy, rotating a steering wheel, or clapping your hands. We create educational and recreational experiences for kids, which promote exploration and discovery, while feeling intuitive and safe. In each experience objects have a unique appearance and behavior e.g. in an electric circuits lab toy blocks serve as switches, batteries and light bulbs.We conducted a user study with children ages 5-11, who experienced TactileVR and interacted with virtual proxies of physical objects. Children took instantly to the TactileVR environment, intuitively discovered a variety of interactions, and completed tasks faster than with non-tactile virtual objects. Moreover, the presence of physical toys created the opportunity for collaborative play, even when only some of the kids were using a VR headset.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133699321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1109/ISMAR-Adjunct.2016.0111
Haomin Liu, Guofeng Zhang, H. Bao
Keyframe-based SLAM has achieved great success in terms of accuracy, efficiency and scalability. However, due to parallax requirement and delay of map expansion, traditional keyframe-based methods easily encounter the robustness problem in the challenging cases especially for fast motion with strong rotation. For AR applications in practice, these challenging cases are easily encountered, since a home user may not carefully move the camera to avoid potential problems. With the above motivation, in this paper, we present RKSLAM, a robust keyframe-based monocular SLAM system that can reliably handle fast motion and strong rotation, ensuring good AR experiences. First, we propose a novel multihomography based feature tracking method which is robust and efficient for fast motion and strong rotation. Based on it, we propose a real-time local map expansion scheme to triangulate the observed 3D points immediately without delay. A sliding-window based camera pose optimization framework is proposed, which imposes the motion prior constraints between consecutive frames through simulated or real IMU data. Qualitative and quantitative comparisons with the state-of-the-art methods, and an AR application on mobile devices demonstrate the effectiveness of the proposed approach.
{"title":"Robust Keyframe-based Monocular SLAM for Augmented Reality","authors":"Haomin Liu, Guofeng Zhang, H. Bao","doi":"10.1109/ISMAR-Adjunct.2016.0111","DOIUrl":"https://doi.org/10.1109/ISMAR-Adjunct.2016.0111","url":null,"abstract":"Keyframe-based SLAM has achieved great success in terms of accuracy, efficiency and scalability. However, due to parallax requirement and delay of map expansion, traditional keyframe-based methods easily encounter the robustness problem in the challenging cases especially for fast motion with strong rotation. For AR applications in practice, these challenging cases are easily encountered, since a home user may not carefully move the camera to avoid potential problems. With the above motivation, in this paper, we present RKSLAM, a robust keyframe-based monocular SLAM system that can reliably handle fast motion and strong rotation, ensuring good AR experiences. First, we propose a novel multihomography based feature tracking method which is robust and efficient for fast motion and strong rotation. Based on it, we propose a real-time local map expansion scheme to triangulate the observed 3D points immediately without delay. A sliding-window based camera pose optimization framework is proposed, which imposes the motion prior constraints between consecutive frames through simulated or real IMU data. Qualitative and quantitative comparisons with the state-of-the-art methods, and an AR application on mobile devices demonstrate the effectiveness of the proposed approach.","PeriodicalId":146808,"journal":{"name":"2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)","volume":"131 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121563455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：