Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687155
Joshitha Ravishankar, Mansi Sharma, Sally Khaidem
Emerging computational light field displays are a suitable choice for realistic presentation of 3D scenes on autostereoscopic glasses-free platforms. However, the enormous size of light field limits their utilization for streaming and 3D display applications. In this paper, we propose a novel representation, coding and streaming scheme for dynamic light fields based on a novel Hybrid Tucker TensorSketch Vector Quantization (HTTSVQ) algorithm. A dynamic light field can be generated from a static light field to capture a moving 3D scene. We acquire images through different coded aperture patterns for a dynamic light field and perform their low-rank approximation using our HTTSVQ scheme, followed by encoding with High Efficiency Video Coding (HEVC). The proposed single pass coding scheme can incrementally handle tensor elements and thus enables to stream and compress light field data without the need to store it in full. Additional encoding of low-rank approximated acquired images by HEVC eliminates intra-frame, inter-frame and intrinsic redundancies in light field data. Comparison with state-of-the-art coders HEVC and its multi-view extension (MV-HEVC) exhibits superior compression performance of the proposed scheme for real-world light fields.
{"title":"A Novel Compression Scheme Based on Hybrid Tucker-Vector Quantization Via Tensor Sketching for Dynamic Light Fields Acquired Through Coded Aperture Camera","authors":"Joshitha Ravishankar, Mansi Sharma, Sally Khaidem","doi":"10.1109/IC3D53758.2021.9687155","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687155","url":null,"abstract":"Emerging computational light field displays are a suitable choice for realistic presentation of 3D scenes on autostereoscopic glasses-free platforms. However, the enormous size of light field limits their utilization for streaming and 3D display applications. In this paper, we propose a novel representation, coding and streaming scheme for dynamic light fields based on a novel Hybrid Tucker TensorSketch Vector Quantization (HTTSVQ) algorithm. A dynamic light field can be generated from a static light field to capture a moving 3D scene. We acquire images through different coded aperture patterns for a dynamic light field and perform their low-rank approximation using our HTTSVQ scheme, followed by encoding with High Efficiency Video Coding (HEVC). The proposed single pass coding scheme can incrementally handle tensor elements and thus enables to stream and compress light field data without the need to store it in full. Additional encoding of low-rank approximated acquired images by HEVC eliminates intra-frame, inter-frame and intrinsic redundancies in light field data. Comparison with state-of-the-art coders HEVC and its multi-view extension (MV-HEVC) exhibits superior compression performance of the proposed scheme for real-world light fields.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133077610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687222
P. A. Kara, Mary Guindy, T. Balogh, Anikó Simon
As the research efforts and development processes behind light field visualization technologies advance, potential novel use cases emerge. These contexts of light field display utilization fundamentally depend on the distance of observation, due to the sheer technological nature of such glasses-free 3D systems. Yet, at the time of this paper, the number of works in the scientific literature that address viewing distance is rather limited, focusing solely on 3D visual experience based on angular density. Thus far, the personal preference of observers regarding viewing distance has not been considered by studies. Furthermore, the upcoming standardization efforts also necessitate research on the topic in order to coherently unify the methodologies of subjective tests. In this paper, we investigate the perceptually-supported and the subjectively-preferred viewing distance of light field visualization. We carried out a series of tests on multiple projection-based light field displays to study these distances, with the separate involvement of experts and regular test participants.
{"title":"The Perceptually-Supported and the Subjectively-Preferred Viewing Distance of Projection-Based Light Field Displays","authors":"P. A. Kara, Mary Guindy, T. Balogh, Anikó Simon","doi":"10.1109/IC3D53758.2021.9687222","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687222","url":null,"abstract":"As the research efforts and development processes behind light field visualization technologies advance, potential novel use cases emerge. These contexts of light field display utilization fundamentally depend on the distance of observation, due to the sheer technological nature of such glasses-free 3D systems. Yet, at the time of this paper, the number of works in the scientific literature that address viewing distance is rather limited, focusing solely on 3D visual experience based on angular density. Thus far, the personal preference of observers regarding viewing distance has not been considered by studies. Furthermore, the upcoming standardization efforts also necessitate research on the topic in order to coherently unify the methodologies of subjective tests. In this paper, we investigate the perceptually-supported and the subjectively-preferred viewing distance of light field visualization. We carried out a series of tests on multiple projection-based light field displays to study these distances, with the separate involvement of experts and regular test participants.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131164471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687239
Hendrik Lievens, Maarten Wijnants, Brent Zoomers, J. Put, Nick Michiels, P. Quax, W. Lamotte
Static light fields are an image-based technology that allow for the photorealistic representation of inanimate objects and scenes in virtual environments. As such, static light fields have application opportunities in heterogeneous domains, including education, cultural heritage and entertainment. This paper contributes the design, implementation and performance evaluation of a web-based static light field consumption system. The proposed system allows static light field datasets to be adaptively streamed over the network and then to be visualized in a vanilla web browser. The performance evaluation results prove that real-time consumption of static light fields at AR/VR-compatible framerates of 90 FPS or more is feasible on commercial off-the-shelf hardware. Given the ubiquitous availability of web browsers on modern consumption devices (PCs, smart TVs, Head Mounted Displays, . . . ), our work is intended to significantly improve the accessibility and exploitation of static light field technology. The JavaScript client code is open-sourced to maximize our work’s impact.
{"title":"Adaptive Streaming and Rendering of Static Light Fields in the Web Browser","authors":"Hendrik Lievens, Maarten Wijnants, Brent Zoomers, J. Put, Nick Michiels, P. Quax, W. Lamotte","doi":"10.1109/IC3D53758.2021.9687239","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687239","url":null,"abstract":"Static light fields are an image-based technology that allow for the photorealistic representation of inanimate objects and scenes in virtual environments. As such, static light fields have application opportunities in heterogeneous domains, including education, cultural heritage and entertainment. This paper contributes the design, implementation and performance evaluation of a web-based static light field consumption system. The proposed system allows static light field datasets to be adaptively streamed over the network and then to be visualized in a vanilla web browser. The performance evaluation results prove that real-time consumption of static light fields at AR/VR-compatible framerates of 90 FPS or more is feasible on commercial off-the-shelf hardware. Given the ubiquitous availability of web browsers on modern consumption devices (PCs, smart TVs, Head Mounted Displays, . . . ), our work is intended to significantly improve the accessibility and exploitation of static light field technology. The JavaScript client code is open-sourced to maximize our work’s impact.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130648272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687169
R. Ruskuls, K. Slics, Sandra Balode, Reinis Ozolins, E. Linina, K. Osmanis, I. Osmanis
In this work we describe the concept of a stereoscopic multi-focal head mounted display for augmented reality applications as a means of mitigating the vergence-accommodation conflict (VAC).Investigated are means of a practical implementation of data transfer between the rendering station and direct control logic within the headset. We rely on a DisplayPort connection as a means to transfer the necessary multi-focal image packets in real time, whilst at the receiving end – the control logic is based on an FPGA architecture responsible for decoding the DisplayPort stream and reformatting data according to the optical layout of the display. Within the design we have chosen to omit local frame buffering which potentially can result in misrepresented data, nevertheless, this approach gains a latency reduction of about 16 ms as opposed to single-frame buffering.
{"title":"Implementation of Multi-Focal Near-Eye Display Architecture: Optimization of Data Path","authors":"R. Ruskuls, K. Slics, Sandra Balode, Reinis Ozolins, E. Linina, K. Osmanis, I. Osmanis","doi":"10.1109/IC3D53758.2021.9687169","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687169","url":null,"abstract":"In this work we describe the concept of a stereoscopic multi-focal head mounted display for augmented reality applications as a means of mitigating the vergence-accommodation conflict (VAC).Investigated are means of a practical implementation of data transfer between the rendering station and direct control logic within the headset. We rely on a DisplayPort connection as a means to transfer the necessary multi-focal image packets in real time, whilst at the receiving end – the control logic is based on an FPGA architecture responsible for decoding the DisplayPort stream and reformatting data according to the optical layout of the display. Within the design we have chosen to omit local frame buffering which potentially can result in misrepresented data, nevertheless, this approach gains a latency reduction of about 16 ms as opposed to single-frame buffering.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129315786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687232
M. Lhuillier
There are several steps to generate a VR environment from images: choose experimental conditions (scene, camera, trajectory, weather), take the images, reconstruct a textured 3D model thanks to a photogrammetry software, and import the 3D model into a game engine. This paper focuses on a post-processing of the photogrammetry step, mostly for outdoor environments that cannot be reconstructed by UAV. Since VR needs a 3D model in a good coordinate system (with a right scale and an axis that is vertical), a simple method is proposed to compute this. In the experiments, we first reconstruct both urban and natural immersive environments by using a helmet-held Gopro Max 360 camera, then import into Unity the 3D models in good coordinate systems, last explore the scenes like a pedestrian thanks to an Oculus Quest.
从图像中生成VR环境有几个步骤:选择实验条件(场景，相机，轨迹，天气)，拍摄图像，通过摄影测量软件重建纹理3D模型，并将3D模型导入游戏引擎。本文主要研究摄影测量步骤的后处理，主要针对无人机无法重建的室外环境。由于VR需要一个良好坐标系下的3D模型(具有正确的比例和垂直的轴)，因此提出了一种简单的计算方法。在实验中，我们首先使用头戴式Gopro Max 360相机重建城市和自然沉浸式环境，然后将良好坐标系下的3D模型导入Unity，最后借助Oculus Quest像行人一样探索场景。
{"title":"From Photogrammetric Reconstruction to Immersive VR Environment","authors":"M. Lhuillier","doi":"10.1109/IC3D53758.2021.9687232","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687232","url":null,"abstract":"There are several steps to generate a VR environment from images: choose experimental conditions (scene, camera, trajectory, weather), take the images, reconstruct a textured 3D model thanks to a photogrammetry software, and import the 3D model into a game engine. This paper focuses on a post-processing of the photogrammetry step, mostly for outdoor environments that cannot be reconstructed by UAV. Since VR needs a 3D model in a good coordinate system (with a right scale and an axis that is vertical), a simple method is proposed to compute this. In the experiments, we first reconstruct both urban and natural immersive environments by using a helmet-held Gopro Max 360 camera, then import into Unity the 3D models in good coordinate systems, last explore the scenes like a pedestrian thanks to an Oculus Quest.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131378459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-08DOI: 10.1109/IC3D53758.2021.9687195
Yupeng Xie, André Souto, Sarah Fachada, Daniele Bonatto, Mehrdad Teratani, G. Lafruit
DIBR (Depth Image Based Rendering) can synthesize Free Navigation virtual views with sparse multiview texture images and corresponding depth maps. There are two ways to obtain depth maps: through software or depth sensors, which is a trade-off between precision versus speed (computational cost and processing time). This article compares the performance of depth maps estimated by MPEG-I’s Depth Estimation Reference Software with that acquired by Kinect Azure. We use IV-PSNR to evaluate their depth maps-based virtual views for the objective comparison. The quality metric with Kinect Azure regularly stay around 32 dB, and its active depth maps yields view synthesis results with better subjective performance in low-textured areas than DERS. Hence, we observe a worthy trade-off in depth performance between Kinect Azure and DERS, but with an advantage of negligible computational cost from the former. We recommend the Kinect Azure for real-time DIBR applications.
DIBR (Depth Image Based Rendering)是一种基于稀疏多视图纹理图像和相应深度图的自由导航虚拟视图合成技术。获得深度图有两种方法:通过软件或深度传感器，这是精度与速度(计算成本和处理时间)之间的权衡。本文比较了MPEG-I深度估计参考软件和Kinect Azure获得的深度图的性能。我们使用IV-PSNR来评估他们基于深度图的虚拟视图，以进行客观比较。Kinect Azure的质量指标通常保持在32db左右，其主动深度图在低纹理区域产生的视图合成结果比DERS具有更好的主观性能。因此，我们观察到Kinect Azure和DERS之间在深度性能上的值得权衡，但前者的计算成本可以忽略不计。我们推荐Kinect Azure用于实时DIBR应用程序。
{"title":"Performance analysis of DIBR-based view synthesis with kinect azure","authors":"Yupeng Xie, André Souto, Sarah Fachada, Daniele Bonatto, Mehrdad Teratani, G. Lafruit","doi":"10.1109/IC3D53758.2021.9687195","DOIUrl":"https://doi.org/10.1109/IC3D53758.2021.9687195","url":null,"abstract":"DIBR (Depth Image Based Rendering) can synthesize Free Navigation virtual views with sparse multiview texture images and corresponding depth maps. There are two ways to obtain depth maps: through software or depth sensors, which is a trade-off between precision versus speed (computational cost and processing time). This article compares the performance of depth maps estimated by MPEG-I’s Depth Estimation Reference Software with that acquired by Kinect Azure. We use IV-PSNR to evaluate their depth maps-based virtual views for the objective comparison. The quality metric with Kinect Azure regularly stay around 32 dB, and its active depth maps yields view synthesis results with better subjective performance in low-textured areas than DERS. Hence, we observe a worthy trade-off in depth performance between Kinect Azure and DERS, but with an advantage of negligible computational cost from the former. We recommend the Kinect Azure for real-time DIBR applications.","PeriodicalId":382937,"journal":{"name":"2021 International Conference on 3D Immersion (IC3D)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133143852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：