Pub Date : 2019-12-01DOI: 10.1109/ic3d48390.2019.8975898
{"title":"IC3D 2019 Message from General Chair","authors":"","doi":"10.1109/ic3d48390.2019.8975898","DOIUrl":"https://doi.org/10.1109/ic3d48390.2019.8975898","url":null,"abstract":"","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125516938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975999
Xudong Fan, Daniele Bonatto, G. Lafruit
Face swapping in videos usually has strong entertainment applications. Deep Fakes (in Faces) are a recent topic in deep learning where the main idea is to substitute the face of a person in a video with the face of another person. But one of the drawbacks of the method is that between two successive frames there are inconsistencies between the faces, such as changing face color, flickering or eyebrows that change. In this paper, we propose a convolutional neural network for swapping faces based on two autoencoders which share the same encoder. In this network, the encoder can distinguish and extract important features of faces, including facial expressions and poses; the decoders will then reconstruct faces according to these features. First, we will generate datasets of faces respectively for person A and person B. Secondly, the local information of two faces is sent to the network to get the model; after the training process, we can use the model to reconstruct the corresponding face of person B when the input is one face of person A. Afterwards, we build a binary mask to select the face area and transfer color from the source face to the target face. Finally, we only need to use a seamless clone to merge the new faces back into the source frames to create a fake video. The experimental results show that the quality of the fake videos is improved significantly.
视频中的人脸交换通常具有很强的娱乐应用。Deep Fakes (in Faces)是深度学习领域最近的一个话题，其主要思想是用另一个人的脸代替视频中的人的脸。但该方法的缺点之一是，在两个连续的帧之间，面部之间存在不一致，例如面部颜色的变化，闪烁或眉毛的变化。在本文中，我们提出了一种基于两个共享同一编码器的自编码器交换人脸的卷积神经网络。在该网络中，编码器可以区分和提取人脸的重要特征，包括面部表情和姿势;然后，解码器将根据这些特征重建人脸。首先，我们将分别生成A人和b人的人脸数据集，然后将两张人脸的局部信息发送到网络中得到模型;训练过程结束后，当输入是a人的一张脸时，我们可以使用该模型重构出B人对应的脸。然后，我们构建一个二值掩码来选择人脸区域，并将源人脸的颜色转移到目标人脸。最后，我们只需要使用无缝克隆将新面孔合并回源帧以创建假视频。实验结果表明，伪视频的质量得到了显著提高。
{"title":"Consistent Long Sequences Deep Faces","authors":"Xudong Fan, Daniele Bonatto, G. Lafruit","doi":"10.1109/IC3D48390.2019.8975999","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975999","url":null,"abstract":"Face swapping in videos usually has strong entertainment applications. Deep Fakes (in Faces) are a recent topic in deep learning where the main idea is to substitute the face of a person in a video with the face of another person. But one of the drawbacks of the method is that between two successive frames there are inconsistencies between the faces, such as changing face color, flickering or eyebrows that change. In this paper, we propose a convolutional neural network for swapping faces based on two autoencoders which share the same encoder. In this network, the encoder can distinguish and extract important features of faces, including facial expressions and poses; the decoders will then reconstruct faces according to these features. First, we will generate datasets of faces respectively for person A and person B. Secondly, the local information of two faces is sent to the network to get the model; after the training process, we can use the model to reconstruct the corresponding face of person B when the input is one face of person A. Afterwards, we build a binary mask to select the face area and transfer color from the source face to the target face. Finally, we only need to use a seamless clone to merge the new faces back into the source frames to create a fake video. The experimental results show that the quality of the fake videos is improved significantly.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"130 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115254243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975992
Mansi Sharma, Gowtham Ragavan, B. Arathi
This paper presents a new depth-image-based rendering algorithm for free-viewpoint 3DTV applications. The cracks, holes, ghost countors caused by visibility, disocclusion, resampling problems associated with 3D warping lead to serious rendering artifacts in synthesized virtual views. This challenging problem of hole filling is formulated as an algebraic matrix completion problem on a higher dimensional space of monomial features described by a novel variety model. The high-level idea of this work is to exploit the linear or nonlinear structures of the data and interpolate missing values by solving algebraic varieties associated with Hankel matrices as a member of Krylov subspace. The proposed model effectively handles artifacts appear in wide-baseline spatial view interpolation and arbitrary camera movements. Our model has a low runtime and results excel with state-of-the-art methods in quantitative and qualitative evaluation.
{"title":"A Novel Algebaric Variety Based Model for High Quality Free-Viewpoint View Synthesis on a Krylov Subspace","authors":"Mansi Sharma, Gowtham Ragavan, B. Arathi","doi":"10.1109/IC3D48390.2019.8975992","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975992","url":null,"abstract":"This paper presents a new depth-image-based rendering algorithm for free-viewpoint 3DTV applications. The cracks, holes, ghost countors caused by visibility, disocclusion, resampling problems associated with 3D warping lead to serious rendering artifacts in synthesized virtual views. This challenging problem of hole filling is formulated as an algebraic matrix completion problem on a higher dimensional space of monomial features described by a novel variety model. The high-level idea of this work is to exploit the linear or nonlinear structures of the data and interpolate missing values by solving algebraic varieties associated with Hankel matrices as a member of Krylov subspace. The proposed model effectively handles artifacts appear in wide-baseline spatial view interpolation and arbitrary camera movements. Our model has a low runtime and results excel with state-of-the-art methods in quantitative and qualitative evaluation.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"123 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116729198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975997
P. A. Kara, R. R. Tamboli, A. Cserkaszky, A. Barsi, Anikó Simon, Agnes Kusz, L. Bokor, M. Martini
Light field displays offer glasses-free 3D visualization, as observers do not need any viewing device to see the content in 3D. The angular resolution of such displays not only determines the achievable smoothness of the parallax effect, but also shapes the valid viewing area of light field visualization; higher angular resolutions support greater viewing distances. Therefore, the binocular disparity of a light field display with a given angular resolution lessens, fades away as the viewing distance increases, and the once true 3D visualization slowly becomes perceptually equivalent to a common 2D projection. However, as the current use case scenarios of light field technology define relatively close observations, this topic is rather under-investigated. In this paper, we address the binocular disparity of projection-based light field displays. The results of objective and subjective studies are presented, in which multiple viewing distances were used to evaluate binocular disparity. Beyond the separate models, the paper analyzes the correlations between them and discusses potential applications for future use cases.
{"title":"Objective and Subjective Assessment of Binocular Disparity for Projection-Based Light Field Displays","authors":"P. A. Kara, R. R. Tamboli, A. Cserkaszky, A. Barsi, Anikó Simon, Agnes Kusz, L. Bokor, M. Martini","doi":"10.1109/IC3D48390.2019.8975997","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975997","url":null,"abstract":"Light field displays offer glasses-free 3D visualization, as observers do not need any viewing device to see the content in 3D. The angular resolution of such displays not only determines the achievable smoothness of the parallax effect, but also shapes the valid viewing area of light field visualization; higher angular resolutions support greater viewing distances. Therefore, the binocular disparity of a light field display with a given angular resolution lessens, fades away as the viewing distance increases, and the once true 3D visualization slowly becomes perceptually equivalent to a common 2D projection. However, as the current use case scenarios of light field technology define relatively close observations, this topic is rather under-investigated. In this paper, we address the binocular disparity of projection-based light field displays. The results of objective and subjective studies are presented, in which multiple viewing distances were used to evaluate binocular disparity. Beyond the separate models, the paper analyzes the correlations between them and discusses potential applications for future use cases.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123849966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8976001
I. B. Ip, C. Lunghi, U. Emir, A. Parker, H. Bridge
Although our view of the world looks singular, it is combined from each eye’s separate retinal image. If the balanced input between eyes is disrupted during early childhood, visual acuity and stereoscopic depth perception are impaired. This is because one eye dominates over the other, causing a neurological condition called ‘amblyopia’ [1]. In the normal, healthy visual system, the balance between eyes can be determined using various methods to provide a measure of ‘eye dominance’. Eye dominance is the preference for using image from one eye over another [2], suggesting that the visual system applies different weights upon their input. Hence, eye dominance is relevant for understanding the mechanisms underlying binocular vision. As an investigative strategy to understand the binocular visual system in health in disease, we want to characterize eye dominance in the normal visual system. This information can then be used to serve as a baseline to compare to extreme eye dominance in ‘amblyopia’. Specifically, we ask to which degree variations in eye dominance are related to visual cortex concentrations of major excitatory neurotransmitter and metabolite glutamate (‘Glu’) and inhibitory neurotransmitter γ-aminobutyric acid (‘GABA’). Their relationship is formalised as the ‘Glu/GABA’ ratio. 13 participants took part in a 1-h psychophysical experiment to quantify eye dominance and a separate 1.5-h 7-Tesla MRI brain scan to measure hemodynamic and neurochemical responses during visual stimulation. The degree of eye dominance was predicted by the inter-ocular difference in V1 Glu/GABA balance. Stronger eye dominance correlated with an increase in inhibition during dominant relative to non-dominant eye viewing (r = −0.647, p = 0.023). In contrast the hemodynamic response, measured with functional magnetic resonance imaging, did not correlate with eye dominance. Our findings suggest that normally occurring eye dominance is associated with the balance of neurochemicals in the early visual cortex.
虽然我们对世界的看法看起来是单一的，但它是由每只眼睛单独的视网膜图像组合而成的。如果在儿童早期眼睛之间的平衡输入被破坏，视觉敏锐度和立体深度感知就会受损。这是因为一只眼睛控制了另一只眼睛，导致了一种称为“弱视”的神经系统疾病[1]。在正常、健康的视觉系统中，眼睛之间的平衡可以通过各种方法来确定，以提供“眼睛优势”的衡量标准。眼优势是对一只眼睛的图像的偏好[2]，这表明视觉系统对它们的输入施加了不同的权重。因此，眼优势与理解双眼视觉的机制是相关的。作为一种研究策略来了解健康疾病中的双眼视觉系统，我们想要表征正常视觉系统中的眼优势。这些信息可以用来作为基线，与“弱视”的极端眼睛优势进行比较。具体来说，我们想知道眼睛优势的变化在多大程度上与视觉皮层主要兴奋性神经递质和代谢物谷氨酸(Glu)和抑制性神经递质γ-氨基丁酸(GABA)的浓度有关。它们的关系被形式化为“Glu/GABA”比率。13名参与者参加了一个1小时的心理物理实验，以量化眼睛优势，并进行了一个1.5小时的7特斯拉MRI脑部扫描，以测量视觉刺激时的血液动力学和神经化学反应。眼优势程度可通过眼间V1 Glu/GABA平衡差异来预测。相对于非优势眼，更强的优势眼与优势眼观看时的抑制增加相关(r = - 0.647, p = 0.023)。相比之下，用功能性磁共振成像测量的血流动力学反应与眼优势无关。我们的研究结果表明，正常发生的眼优势与早期视觉皮层中神经化学物质的平衡有关。
{"title":"Relating Eye Dominance to Neurochemistry in the Human Visual Cortex Using Ultra High Field 7-Tesla MR Spectroscopy","authors":"I. B. Ip, C. Lunghi, U. Emir, A. Parker, H. Bridge","doi":"10.1109/IC3D48390.2019.8976001","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8976001","url":null,"abstract":"Although our view of the world looks singular, it is combined from each eye’s separate retinal image. If the balanced input between eyes is disrupted during early childhood, visual acuity and stereoscopic depth perception are impaired. This is because one eye dominates over the other, causing a neurological condition called ‘amblyopia’ [1]. In the normal, healthy visual system, the balance between eyes can be determined using various methods to provide a measure of ‘eye dominance’. Eye dominance is the preference for using image from one eye over another [2], suggesting that the visual system applies different weights upon their input. Hence, eye dominance is relevant for understanding the mechanisms underlying binocular vision. As an investigative strategy to understand the binocular visual system in health in disease, we want to characterize eye dominance in the normal visual system. This information can then be used to serve as a baseline to compare to extreme eye dominance in ‘amblyopia’. Specifically, we ask to which degree variations in eye dominance are related to visual cortex concentrations of major excitatory neurotransmitter and metabolite glutamate (‘Glu’) and inhibitory neurotransmitter γ-aminobutyric acid (‘GABA’). Their relationship is formalised as the ‘Glu/GABA’ ratio. 13 participants took part in a 1-h psychophysical experiment to quantify eye dominance and a separate 1.5-h 7-Tesla MRI brain scan to measure hemodynamic and neurochemical responses during visual stimulation. The degree of eye dominance was predicted by the inter-ocular difference in V1 Glu/GABA balance. Stronger eye dominance correlated with an increase in inhibition during dominant relative to non-dominant eye viewing (r = −0.647, p = 0.023). In contrast the hemodynamic response, measured with functional magnetic resonance imaging, did not correlate with eye dominance. Our findings suggest that normally occurring eye dominance is associated with the balance of neurochemicals in the early visual cortex.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121591116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975996
Ágota Faluvégi, Quentin Bolsée, S. Nedevschi, V. Dădârlat, A. Munteanu
Depth estimation has always been a great challenge in the field of computer vision and machine learning. There is a rich literature focusing on depth estimation in stereo vision or in monocular imaging, while the domain of depth estimation in light field images is still in its infancy. The paper proposes a fully convolutional 3D neural network that estimates the disparity in light field images. The proposed method is parametric as it is able to adapt to input images of arbitrary size and it is lightweight and less prone to overfitting thanks to its fully convolutional nature. The experiments reveal competitive results against the state of the art, demonstrating the potential offered by deep learning solutions for disparity estimation in light field images.
{"title":"A 3D Convolutional Neural Network for Light Field Depth Estimation","authors":"Ágota Faluvégi, Quentin Bolsée, S. Nedevschi, V. Dădârlat, A. Munteanu","doi":"10.1109/IC3D48390.2019.8975996","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975996","url":null,"abstract":"Depth estimation has always been a great challenge in the field of computer vision and machine learning. There is a rich literature focusing on depth estimation in stereo vision or in monocular imaging, while the domain of depth estimation in light field images is still in its infancy. The paper proposes a fully convolutional 3D neural network that estimates the disparity in light field images. The proposed method is parametric as it is able to adapt to input images of arbitrary size and it is lightweight and less prone to overfitting thanks to its fully convolutional nature. The experiments reveal competitive results against the state of the art, demonstrating the potential offered by deep learning solutions for disparity estimation in light field images.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133481161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975989
L. André, R. Coutellier
Many questions arise regarding the use of virtual reality (VR) in the naval military field. This is particularly the case from a human factors’ perspective, where system’s usability is supposed to guarantee user’s performance. Cybersickness, resulting from a sensory conflict between visual and vestibular systems, is one of the major limitations to the development of VR. Cybersickness can lead to nausea, oculomotor discomfort and disorientation. The major aim of the current study was to evaluate the efficiency of a remediation system for cybersickness. This system is designed to help remove the sensory conflict, thanks to miniature LED screens placed in the head mounted display (HMD). 18 subjects were confronted with a dynamic environment in VR, equipped with HMDs. Different physiological variables were measured during immersion. Every subject showed effects of cybersickness, starting, on average, after eight minutes of exposure, even if the system may reduce symptoms under certain conditions.
{"title":"Cybersickness and Evaluation of a Remediation System: A Pilot Study","authors":"L. André, R. Coutellier","doi":"10.1109/IC3D48390.2019.8975989","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975989","url":null,"abstract":"Many questions arise regarding the use of virtual reality (VR) in the naval military field. This is particularly the case from a human factors’ perspective, where system’s usability is supposed to guarantee user’s performance. Cybersickness, resulting from a sensory conflict between visual and vestibular systems, is one of the major limitations to the development of VR. Cybersickness can lead to nausea, oculomotor discomfort and disorientation. The major aim of the current study was to evaluate the efficiency of a remediation system for cybersickness. This system is designed to help remove the sensory conflict, thanks to miniature LED screens placed in the head mounted display (HMD). 18 subjects were confronted with a dynamic environment in VR, equipped with HMDs. Different physiological variables were measured during immersion. Every subject showed effects of cybersickness, starting, on average, after eight minutes of exposure, even if the system may reduce symptoms under certain conditions.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131320548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975901
I. Schiopu, Patrice Rondao-Alface, A. Munteanu
The paper proposes a novel frame-wise view synthesis method based on convolutional neural networks (CNNs) for wide-baseline light field (LF) camera arrays. A novel neural network architecture that follows a multi-resolution processing paradigm is employed to synthesize an entire view. A novel loss function formulation based on the structural similarity index (SSIM) is proposed. A wide-baseline LF image dataset is generated and employed to train the proposed deep model. The proposed method synthesizes each subaperture image (SAI) from a LF image based on corresponding SAIs from two reference LF images. Experimental results show that the proposed method yields promising results with an average PSNR and SSIM of 34.71 dB and 0.9673 respectively for wide baselines.
{"title":"Frame-Wise CNN-Based View Synthesis for Light Field Camera Arrays","authors":"I. Schiopu, Patrice Rondao-Alface, A. Munteanu","doi":"10.1109/IC3D48390.2019.8975901","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975901","url":null,"abstract":"The paper proposes a novel frame-wise view synthesis method based on convolutional neural networks (CNNs) for wide-baseline light field (LF) camera arrays. A novel neural network architecture that follows a multi-resolution processing paradigm is employed to synthesize an entire view. A novel loss function formulation based on the structural similarity index (SSIM) is proposed. A wide-baseline LF image dataset is generated and employed to train the proposed deep model. The proposed method synthesizes each subaperture image (SAI) from a LF image based on corresponding SAIs from two reference LF images. Experimental results show that the proposed method yields promising results with an average PSNR and SSIM of 34.71 dB and 0.9673 respectively for wide baselines.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122265010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975991
Rodrigo Diaz, Aurela Shehu, I. Feldmann, O. Schreer, P. Eisert
This paper addresses high quality mesh optimization for volumetric video. Real persons are captured with multiple cameras and converted to 3D mesh sequences. These volumetric video assets can be used as dynamic 3D objects in arbitrary 3D rendering engines. In this way, 3D representations of real persons are achieved with a high level of detail and realism. Target use cases are augmented reality, virtual reality and mixed reality applications. However, the final rendering quality strongly depends on the hardware capabilities of the target rendering device. In this context, a novel region dependent mesh refinement approach is presented and evaluated with respect to existing workflows. The proposed approach is used in order to obtain a low overall polygon count while keeping details in semantically important regions such as human faces. It combines conventional 2D skin and face detection algorithms and transfers the results to the 3D domain. Further on, a dedicated camera region selection approach is presented which enhances the sharpness and quality of the resulting 3D texture mappings.
{"title":"Region Dependent Mesh Refinement for Volumetric Video Workflows","authors":"Rodrigo Diaz, Aurela Shehu, I. Feldmann, O. Schreer, P. Eisert","doi":"10.1109/IC3D48390.2019.8975991","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975991","url":null,"abstract":"This paper addresses high quality mesh optimization for volumetric video. Real persons are captured with multiple cameras and converted to 3D mesh sequences. These volumetric video assets can be used as dynamic 3D objects in arbitrary 3D rendering engines. In this way, 3D representations of real persons are achieved with a high level of detail and realism. Target use cases are augmented reality, virtual reality and mixed reality applications. However, the final rendering quality strongly depends on the hardware capabilities of the target rendering device. In this context, a novel region dependent mesh refinement approach is presented and evaluated with respect to existing workflows. The proposed approach is used in order to obtain a low overall polygon count while keeping details in semantically important regions such as human faces. It combines conventional 2D skin and face detection algorithms and transfers the results to the 3D domain. Further on, a dedicated camera region selection approach is presented which enhances the sharpness and quality of the resulting 3D texture mappings.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132621804","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/IC3D48390.2019.8975988
Mansi Sharma, M. S. Venkatesh, Gowtham Ragavan, Rohan Lal
In this work, we proposed a novel depth adaptive tone mapping scheme for stereo HDR imaging and 3D display. We are interested in the case where different exposures are taken from different viewpoints. The scheme employed a new depth-adaptive cross-trilateral filter (DA-CTF) for recovering High Dynamic Range (HDR) images from multiple Low Dynamic Range (LDR) images captured at different exposure levels. Explicitly leveraging additional depth information in the tone mapping operation correctly identify global contrast change and detail visibility change by preserving the edges and reducing halo artifacts in the synthesized 3D views by depth-image-based rendering (DIBR) procedure. The experiments show that the proposed DA-CTF and DIBR scheme outperforms state-of-the-art operators in the enhanced depiction of tone mapped HDR stereo images on LDR displays.
{"title":"A Novel Approach for Multi-View 3D HDR Content Generation via Depth Adaptive Cross Trilateral Tone Mapping","authors":"Mansi Sharma, M. S. Venkatesh, Gowtham Ragavan, Rohan Lal","doi":"10.1109/IC3D48390.2019.8975988","DOIUrl":"https://doi.org/10.1109/IC3D48390.2019.8975988","url":null,"abstract":"In this work, we proposed a novel depth adaptive tone mapping scheme for stereo HDR imaging and 3D display. We are interested in the case where different exposures are taken from different viewpoints. The scheme employed a new depth-adaptive cross-trilateral filter (DA-CTF) for recovering High Dynamic Range (HDR) images from multiple Low Dynamic Range (LDR) images captured at different exposure levels. Explicitly leveraging additional depth information in the tone mapping operation correctly identify global contrast change and detail visibility change by preserving the edges and reducing halo artifacts in the synthesized 3D views by depth-image-based rendering (DIBR) procedure. The experiments show that the proposed DA-CTF and DIBR scheme outperforms state-of-the-art operators in the enhanced depiction of tone mapped HDR stereo images on LDR displays.","PeriodicalId":344706,"journal":{"name":"2019 International Conference on 3D Immersion (IC3D)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128059548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：