Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687162
Herbert Potechius, T. Sikora, S. Knorr
In this paper, we analyse and compare four different color transfer algorithms, which were originally developed for 2D images, for 3D point clouds. In particular, we transfer the color distribution of a given reference 3D point cloud to a source 3D point cloud in order to change the illumination of the scene. Color transfer of 3D models might become an important task in AR and VR applications where e.g., an existing 3D model needs to be updated in real-time according to scene changes like displaced objects and illumination changes. In order to better compare the results of the color transfer algorithms, we created a data set of 3D point clouds consisting of reconstructions of an indoor scene under different lighting conditions, and applied two comparison methods for an objective evaluation, namely histogram comparison and voxel comparison, which will be described in detail in this paper.
{"title":"Color Transfer of 3D Point Clouds For XR Applications","authors":"Herbert Potechius, T. Sikora, S. Knorr","doi":"10.1109/IC3D53758.2021.9687162","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687162","url":null,"abstract":"In this paper, we analyse and compare four different color transfer algorithms, which were originally developed for 2D images, for 3D point clouds. In particular, we transfer the color distribution of a given reference 3D point cloud to a source 3D point cloud in order to change the illumination of the scene. Color transfer of 3D models might become an important task in AR and VR applications where e.g., an existing 3D model needs to be updated in real-time according to scene changes like displaced objects and illumination changes. In order to better compare the results of the color transfer algorithms, we created a data set of 3D point clouds consisting of reconstructions of an indoor scene under different lighting conditions, and applied two comparison methods for an objective evaluation, namely histogram comparison and voxel comparison, which will be described in detail in this paper.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115345602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687256
Decai Chen, Markus Worchel, I. Feldmann, O. Schreer, P. Eisert
In this work, we enhance a professional end-to-end volumetric video production pipeline to achieve high-fidelity human body reconstruction using only passive cameras. While current volumetric video approaches estimate depth maps using traditional stereo matching techniques, we introduce and optimize deep learning-based multi-view stereo networks for depth map estimation in the context of professional volumetric video reconstruction. Furthermore, we propose a novel depth map post-processing approach including filtering and fusion, by taking into account photometric confidence, cross-view geometric consistency, foreground masks as well as camera viewing frustums. We show that our method can generate high levels of geometric detail for reconstructed human bodies.
{"title":"Accurate human body reconstruction for volumetric video","authors":"Decai Chen, Markus Worchel, I. Feldmann, O. Schreer, P. Eisert","doi":"10.1109/IC3D53758.2021.9687256","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687256","url":null,"abstract":"In this work, we enhance a professional end-to-end volumetric video production pipeline to achieve high-fidelity human body reconstruction using only passive cameras. While current volumetric video approaches estimate depth maps using traditional stereo matching techniques, we introduce and optimize deep learning-based multi-view stereo networks for depth map estimation in the context of professional volumetric video reconstruction. Furthermore, we propose a novel depth map post-processing approach including filtering and fusion, by taking into account photometric confidence, cross-view geometric consistency, foreground masks as well as camera viewing frustums. We show that our method can generate high levels of geometric detail for reconstructed human bodies.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"314 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133227230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687182
Mary Guindy, V. K. Adhikarla, P. A. Kara, T. Balogh, Anikó Simon
The reconstruction of high dynamic range (HDR) images from low dynamic range (LDR) images is a challenging task. Multiple algorithms are implemented to perform the reconstruction process for HDR images and videos. These techniques include, but are not limited to reverse tone mapping, computational photography and convolutional neural networks (CNNs). From the aforementioned techniques, CNNs have proven to be the most efficient when tested against conventional 2D images and videos. However, at the time of this paper, applying such CNNs to light field contents have not yet been performed. Light field images impose more challenges and difficulties to the proposed CNNs, as there are multiple images for the creation of a single light field scene. In this paper, we test some of the existing HDR CNNs (ExpandNet, HDR-DeepCNN and DeepHDRVideo) on the Teddy light field image dataset and evaluate their performance using PSNR, SSIM and HDR-VDP 2.2.1. Our work addresses both image and video reconstruction techniques in the context of light field imaging. The results indicate that further modifications to the state-of-the-art reconstruction techniques are required to efficiently accommodate the spatial coherence in light field images.
{"title":"Performance Evaluation of HDR Image Reconstruction Techniques on Light Field Images","authors":"Mary Guindy, V. K. Adhikarla, P. A. Kara, T. Balogh, Anikó Simon","doi":"10.1109/IC3D53758.2021.9687182","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687182","url":null,"abstract":"The reconstruction of high dynamic range (HDR) images from low dynamic range (LDR) images is a challenging task. Multiple algorithms are implemented to perform the reconstruction process for HDR images and videos. These techniques include, but are not limited to reverse tone mapping, computational photography and convolutional neural networks (CNNs). From the aforementioned techniques, CNNs have proven to be the most efficient when tested against conventional 2D images and videos. However, at the time of this paper, applying such CNNs to light field contents have not yet been performed. Light field images impose more challenges and difficulties to the proposed CNNs, as there are multiple images for the creation of a single light field scene. In this paper, we test some of the existing HDR CNNs (ExpandNet, HDR-DeepCNN and DeepHDRVideo) on the Teddy light field image dataset and evaluate their performance using PSNR, SSIM and HDR-VDP 2.2.1. Our work addresses both image and video reconstruction techniques in the context of light field imaging. The results indicate that further modifications to the state-of-the-art reconstruction techniques are required to efficiently accommodate the spatial coherence in light field images.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126691605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687243
Daniele Bonatto, Sarah Fachada, T. Senoh, Guotai Jiang, Xin Jin, G. Lafruit, Mehrdad Teratani
Multi-focused Plenoptic cameras (Plenoptic 2.0) allow the acquisition of the Light-Field of a scene. However, extracting a novel view from the resulting Micro-Lens Array (MLA) image poses several challenges: micro-lenses calibration, noise reduction, patch size (depth) estimation to convert micro-lens image to multi-view images. We propose a novel method to easily find important micro-lenses parameters, avoid the unreliable luminance area, estimate the depth map, and extract sub-aperture images (multiview) for the single- and multi-focused Plenoptic 2.0 camera. Our results demonstrate significant improvement in quality and reduction in computational time compared to the state-of-the-art conversion tool Reference Lenslet content Convertor from MLA image to multiview images.
{"title":"Multiview from micro-lens image of multi-focused plenoptic camera","authors":"Daniele Bonatto, Sarah Fachada, T. Senoh, Guotai Jiang, Xin Jin, G. Lafruit, Mehrdad Teratani","doi":"10.1109/IC3D53758.2021.9687243","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687243","url":null,"abstract":"Multi-focused Plenoptic cameras (Plenoptic 2.0) allow the acquisition of the Light-Field of a scene. However, extracting a novel view from the resulting Micro-Lens Array (MLA) image poses several challenges: micro-lenses calibration, noise reduction, patch size (depth) estimation to convert micro-lens image to multi-view images. We propose a novel method to easily find important micro-lenses parameters, avoid the unreliable luminance area, estimate the depth map, and extract sub-aperture images (multiview) for the single- and multi-focused Plenoptic 2.0 camera. Our results demonstrate significant improvement in quality and reduction in computational time compared to the state-of-the-art conversion tool Reference Lenslet content Convertor from MLA image to multiview images.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"20 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132497895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687271
Balasubramanyam Appina, Mansi Sharma, Santosh Kumar, P. A. Kara, Anikó Simon, Mary Guindy
Numerous stereoscopic 3D movies are released every single year to movie theaters and they evidently generate large revenues. Despite the notable improvements in stereo capturing and 3D video post-production technologies, stereoscopic artefacts continue to appear even in high-budget films. Existing automatic 3D video quality measurement tools can detect distortions in stereoscopic images and videos, but they fail to determine the viewer’s subjective perception of those arte-facts, and how these distortions affect their choices and the overall visual experience. In this paper, we introduce a novel recommendation system for stereoscopic 3D movies based on a latent factor model that meticulously analyzes the viewer’s subjective ratings and the influence of 3D video distortions on their personal preferences. To the best knowledge of the authors, this is definitely a first-of-its-kind model that recommends 3D movies based on quality ratings. It takes the correlation between the viewer’s visual discomfort and the perception of stereoscopic artefacts into account. The proposed model is trained and tested on the benchmark Nama3ds1-cospad1 and LFOVIAS3DPh2 S3D video quality assessment datasets. The experiments highlight the practical efficiency and considerable performance of the resulting matrix-factorization-based recommendation system.
{"title":"Latent Factor Modeling of Perceived Quality for Stereoscopic 3D Video Recommendation","authors":"Balasubramanyam Appina, Mansi Sharma, Santosh Kumar, P. A. Kara, Anikó Simon, Mary Guindy","doi":"10.1109/IC3D53758.2021.9687271","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687271","url":null,"abstract":"Numerous stereoscopic 3D movies are released every single year to movie theaters and they evidently generate large revenues. Despite the notable improvements in stereo capturing and 3D video post-production technologies, stereoscopic artefacts continue to appear even in high-budget films. Existing automatic 3D video quality measurement tools can detect distortions in stereoscopic images and videos, but they fail to determine the viewer’s subjective perception of those arte-facts, and how these distortions affect their choices and the overall visual experience. In this paper, we introduce a novel recommendation system for stereoscopic 3D movies based on a latent factor model that meticulously analyzes the viewer’s subjective ratings and the influence of 3D video distortions on their personal preferences. To the best knowledge of the authors, this is definitely a first-of-its-kind model that recommends 3D movies based on quality ratings. It takes the correlation between the viewer’s visual discomfort and the perception of stereoscopic artefacts into account. The proposed model is trained and tested on the benchmark Nama3ds1-cospad1 and LFOVIAS3DPh2 S3D video quality assessment datasets. The experiments highlight the practical efficiency and considerable performance of the resulting matrix-factorization-based recommendation system.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"2017 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132103173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687263
Sarah Fachada, Daniele Bonatto, Yupeng Xie, Patrice Rondao-Alface, Mehrdad Teratani, G. Lafruit
In the context of the development of MPEG-I standard for immersive video compression ISO/IEC 23090-12 (MIV), the need of handling scenes with non-Lambertian materials arose. This class of material is omnipresent in natural scenes, but violates all the assumptions on which depth image-based rendering (DIBR) is based. In this paper, we present a view-synthesizer designed to handle non-Lambertian objects with DIBR, replacing the classical depth maps by multi-coefficients non-Lambertian maps. We report the results of the exploration experiments on Future MIV designed to test this rendering method against the classical DIBR approaches, and demonstrate promising results on all the tested sequences.
{"title":"Depth Image-Based Rendering of Non-Lambertian Content in MPEG Immersive Video","authors":"Sarah Fachada, Daniele Bonatto, Yupeng Xie, Patrice Rondao-Alface, Mehrdad Teratani, G. Lafruit","doi":"10.1109/IC3D53758.2021.9687263","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687263","url":null,"abstract":"In the context of the development of MPEG-I standard for immersive video compression ISO/IEC 23090-12 (MIV), the need of handling scenes with non-Lambertian materials arose. This class of material is omnipresent in natural scenes, but violates all the assumptions on which depth image-based rendering (DIBR) is based. In this paper, we present a view-synthesizer designed to handle non-Lambertian objects with DIBR, replacing the classical depth maps by multi-coefficients non-Lambertian maps. We report the results of the exploration experiments on Future MIV designed to test this rendering method against the classical DIBR approaches, and demonstrate promising results on all the tested sequences.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125184914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/ic3d53758.2021.9687231
{"title":"[Copyright notice]","authors":"","doi":"10.1109/ic3d53758.2021.9687231","DOIUrl":"https://doi.org/10.1109/ic3d53758.2021.9687231","url":null,"abstract":"","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"195 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134396590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687294
Haoshuo Wang, Colm O. Fearghail, Emin Zerman, Karsten Braungart, A. Smolic, S. Knorr
Due to the character of 360° video, it is often a challenge for filmmakers to guide the attention of users to the region of interest. Visual effects as a type of user guidance is frequently applied to traditional film. Nevertheless, the influence of visual effects in 360° video has been rarely explored. For this reason, the purpose of this paper is to study how four different visual effects, respectively Desaturation, Context-based Darkening, Area Darkening, and Object to Follow, affect visual attention of users in 360° video. Therefore, we performed a subjective test as well as analyzed the saliency maps predicted with a convolutional neural network. In the subjective test, 15 participants were requested to watch four 360° videos, which were implemented with visual effects, while the position of their viewport was recorded. The results of this work are compared to earlier research on the same videos without visual effects. We show that Area Darkening has the best effect on guiding the visual attention, Context-based Darkening makes the best contribution on enhancing the saliency of the region of interest, while Desaturation has nearly no effect for user guidance and does not change the saliency of the videos. A Logo as Object to Follow create a new salient area, while the predicted saliency of areas apart from the Logo remains the same.
{"title":"Visual Attention Analysis and User Guidance in Cinematic VR Film","authors":"Haoshuo Wang, Colm O. Fearghail, Emin Zerman, Karsten Braungart, A. Smolic, S. Knorr","doi":"10.1109/IC3D53758.2021.9687294","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687294","url":null,"abstract":"Due to the character of 360° video, it is often a challenge for filmmakers to guide the attention of users to the region of interest. Visual effects as a type of user guidance is frequently applied to traditional film. Nevertheless, the influence of visual effects in 360° video has been rarely explored. For this reason, the purpose of this paper is to study how four different visual effects, respectively Desaturation, Context-based Darkening, Area Darkening, and Object to Follow, affect visual attention of users in 360° video. Therefore, we performed a subjective test as well as analyzed the saliency maps predicted with a convolutional neural network. In the subjective test, 15 participants were requested to watch four 360° videos, which were implemented with visual effects, while the position of their viewport was recorded. The results of this work are compared to earlier research on the same videos without visual effects. We show that Area Darkening has the best effect on guiding the visual attention, Context-based Darkening makes the best contribution on enhancing the saliency of the region of interest, while Desaturation has nearly no effect for user guidance and does not change the saliency of the videos. A Logo as Object to Follow create a new salient area, while the predicted saliency of areas apart from the Logo remains the same.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122260882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687218
Lucas Pometti, Matthieu Fradet, P. Hirtzlin, Pierrick Jouet
In this paper we present our on-going work on light source estimation in room-scale scenes for more photorealistic experiences. Our unique input is an up-to-date textured 3D mesh of a real uncontrolled environment obtained using a consumer mobile device. We base our approach on the detection of real shadows in a single RGB-D image rendered for a top viewpoint. Contrary to prior art, our approach does not consider any object-based segmentation, neither simplifying assumptions on the scene geometry or on poorly textured surfaces. The 3D locations of light sources are automatically estimated, and for now, the lighting model is completed with intensity values obtained interactively through a GUI displaying augmentations on the scanned scene. This lighting model can then be reused to light the MR scene coherently during mobile experiences. Results on various indoor and outdoor scenes show the beginnings of a promising work. To illustrate the complexity of the problem and to make the community aware of the importance of a correct lighting on user perception, we also fairly show how slightly inaccurate light estimation results or incomplete geometry knowledge can go completely unnoticed in some simple cases but can also deeply impact the final rendering photorealism in some other cases.
{"title":"3D location estimation of light sources in room-scale scenes","authors":"Lucas Pometti, Matthieu Fradet, P. Hirtzlin, Pierrick Jouet","doi":"10.1109/IC3D53758.2021.9687218","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687218","url":null,"abstract":"In this paper we present our on-going work on light source estimation in room-scale scenes for more photorealistic experiences. Our unique input is an up-to-date textured 3D mesh of a real uncontrolled environment obtained using a consumer mobile device. We base our approach on the detection of real shadows in a single RGB-D image rendered for a top viewpoint. Contrary to prior art, our approach does not consider any object-based segmentation, neither simplifying assumptions on the scene geometry or on poorly textured surfaces. The 3D locations of light sources are automatically estimated, and for now, the lighting model is completed with intensity values obtained interactively through a GUI displaying augmentations on the scanned scene. This lighting model can then be reused to light the MR scene coherently during mobile experiences. Results on various indoor and outdoor scenes show the beginnings of a promising work. To illustrate the complexity of the problem and to make the community aware of the importance of a correct lighting on user perception, we also fairly show how slightly inaccurate light estimation results or incomplete geometry knowledge can go completely unnoticed in some simple cases but can also deeply impact the final rendering photorealism in some other cases.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132709699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687257
Charles Hamesse, B. Pairet, Rihab Lahouli, Timothée Fréville, R. Haelterman
Using Augmented Reality (AR) technology for Air Traffic Control (ATC) holds great promise but comes with a number of technical challenges. In addition to displaying the position of surrounding aircraft, a zoomed-in view of a certain aircraft captured with a Pan-Tilt-Zoom (PTZ) camera is also very useful in practice. This is to allow the ATC officer to perform the visual checks that they typically do with large binoculars, directly with the AR headset. Therefore, the PTZ camera has to be able to track the aircraft in a fast and robust manner to produce images suitable to be projected on the AR headset. Unfortunately, PTZ tracking algorithms are notoriously hard to implement, since the captured images depend on the PTZ controls, which depend on the outputs of the tracking algorithm, which depend on the images. In this paper, we describe our generic framework which leverages 3D simulation to design PTZ tracking algorithms and offer an in-depth explanation of how we use it in the context of AR ATC.
{"title":"Simulation of Pan-Tilt-Zoom Tracking for Augmented Reality Air Traffic Control","authors":"Charles Hamesse, B. Pairet, Rihab Lahouli, Timothée Fréville, R. Haelterman","doi":"10.1109/IC3D53758.2021.9687257","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687257","url":null,"abstract":"Using Augmented Reality (AR) technology for Air Traffic Control (ATC) holds great promise but comes with a number of technical challenges. In addition to displaying the position of surrounding aircraft, a zoomed-in view of a certain aircraft captured with a Pan-Tilt-Zoom (PTZ) camera is also very useful in practice. This is to allow the ATC officer to perform the visual checks that they typically do with large binoculars, directly with the AR headset. Therefore, the PTZ camera has to be able to track the aircraft in a fast and robust manner to produce images suitable to be projected on the AR headset. Unfortunately, PTZ tracking algorithms are notoriously hard to implement, since the captured images depend on the PTZ controls, which depend on the outputs of the tracking algorithm, which depend on the images. In this paper, we describe our generic framework which leverages 3D simulation to design PTZ tracking algorithms and offer an in-depth explanation of how we use it in the context of AR ATC.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124469698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：