Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069656
S. Shimizu, H. Kimata, Y. Ohtani
This paper presents a real-time video-based rendering system that uses multiview video data with depth representation for free-viewpoint navigation. The proposed rendering algorithm not only achieves high quality rendering but also increases viewpoint flexibility to cover viewpoints that do not lie on the camera baselines. The proposed system achieves real-time decoding of multiple videos and depth maps that are encoded by the H.264/AVC Multiview Video Coding Extension on a regular CPU. The rendering process is fully implemented on a commercial GPU. A performance evaluation shows that our system can generate XGA free-viewpoint images at 30 fps.
{"title":"Real-time free-viewpoint viewer from multiview video plus depth representation coded by H.264/AVC MVC extension","authors":"S. Shimizu, H. Kimata, Y. Ohtani","doi":"10.1109/3DTV.2009.5069656","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069656","url":null,"abstract":"This paper presents a real-time video-based rendering system that uses multiview video data with depth representation for free-viewpoint navigation. The proposed rendering algorithm not only achieves high quality rendering but also increases viewpoint flexibility to cover viewpoints that do not lie on the camera baselines. The proposed system achieves real-time decoding of multiple videos and depth maps that are encoded by the H.264/AVC Multiview Video Coding Extension on a regular CPU. The rendering process is fully implemented on a commercial GPU. A performance evaluation shows that our system can generate XGA free-viewpoint images at 30 fps.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"229 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127532677","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A multiview stereo video FGS (Fine Granular Scalability) scalable scheme is presented in this paper. The similarity among adjacent views is fully utilized, A tradeoff scheme is presented in order to adapt to different demands of Quality First (QF) and View First (VF) of the decoder. The scheme is composed of three cases: I, P, B frame. The middle view is encoded as the basic layer, while the other views are predicted from the partly retrieved FGS enhancement layers of adjacent views. The FGS enhancement layer of the current view is generated based on that. Experimental results show that the presented scheme is of more flexible and extensive scalable characteristic, which could better adapt different demands on view image quality and stereo immersion of different users.
{"title":"An improved multiview stereo video FGS scalable scheme","authors":"Lei Yang, Xiaowei Song, Chunping Hou, Jichang Guo, Sumei Li, Yuan Zhou","doi":"10.1109/3DTV.2009.5069658","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069658","url":null,"abstract":"A multiview stereo video FGS (Fine Granular Scalability) scalable scheme is presented in this paper. The similarity among adjacent views is fully utilized, A tradeoff scheme is presented in order to adapt to different demands of Quality First (QF) and View First (VF) of the decoder. The scheme is composed of three cases: I, P, B frame. The middle view is encoded as the basic layer, while the other views are predicted from the partly retrieved FGS enhancement layers of adjacent views. The FGS enhancement layer of the current view is generated based on that. Experimental results show that the presented scheme is of more flexible and extensive scalable characteristic, which could better adapt different demands on view image quality and stereo immersion of different users.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114416001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069669
P. Zanuttigh, G. Cortelazzo
This paper presents a novel strategy for the compression of depth maps. The proposed scheme starts with a segmentation step which identifies and extracts edges and main objects, then it introduces an efficient compression strategy for the segmented regions' shape. In the subsequent step a novel algorithm is used to predict the surface shape from the segmented regions and a set of regularly spaced samples. Finally the few prediction residuals are efficiently compressed using standard image compression techniques. Experimental results show that the proposed scheme not only offers a significant gain over JPEG2000 on various types of depth maps but also produces depth maps without edge artifacts particularly suited to 3D warping and free viewpoint video applications.
{"title":"Compression of depth information for 3D rendering","authors":"P. Zanuttigh, G. Cortelazzo","doi":"10.1109/3DTV.2009.5069669","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069669","url":null,"abstract":"This paper presents a novel strategy for the compression of depth maps. The proposed scheme starts with a segmentation step which identifies and extracts edges and main objects, then it introduces an efficient compression strategy for the segmented regions' shape. In the subsequent step a novel algorithm is used to predict the surface shape from the segmented regions and a set of regularly spaced samples. Finally the few prediction residuals are efficiently compressed using standard image compression techniques. Experimental results show that the proposed scheme not only offers a significant gain over JPEG2000 on various types of depth maps but also produces depth maps without edge artifacts particularly suited to 3D warping and free viewpoint video applications.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128980970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069638
Cédric Niquin, S. Prévost, Y. Rémion
We present an offline method for stereo matching using a large number of views. Our method is based on occlusions detection. It is composed of two steps, one global and one local. In the first step we formulate an energy function that handles data, occlusions, and smooth terms through a global graph-cuts optimization. In our second step we introduce a local cost that handles occlusions from the first step in order to refine the result. This cost takes advantage of both the multi-view aspect and the occlusions. The experimental results show how our algorithm joins the advantages of both global and local methods, and how much it is accurate on boundaries detection and on details.
{"title":"Accurate multi-view depth reconstruction with occlusions handling","authors":"Cédric Niquin, S. Prévost, Y. Rémion","doi":"10.1109/3DTV.2009.5069638","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069638","url":null,"abstract":"We present an offline method for stereo matching using a large number of views. Our method is based on occlusions detection. It is composed of two steps, one global and one local. In the first step we formulate an energy function that handles data, occlusions, and smooth terms through a global graph-cuts optimization. In our second step we introduce a local cost that handles occlusions from the first step in order to refine the result. This cost takes advantage of both the multi-view aspect and the occlusions. The experimental results show how our algorithm joins the advantages of both global and local methods, and how much it is accurate on boundaries detection and on details.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"187 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116322117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069663
C. Weigel, S. Schwarz, T. Korn, Martin Wallebohr
We present a system for rendering free viewpoint video from data acquired by one or more stereo camera pairs in advance. The free viewpoint video can be observed standalone or shown embedded in a synthetic computer graphics scene. Compared to state-of-the art free viewpoint video applications less cameras are required. The system is scalable in terms of adding more stereo pairs in order to increase the viewing latitude around the object and is therefore adaptable to different kinds of application such as quality assessment tasks or virtual fairs. The main contribution of this paper are i) the scalable extension of the system by additional stereo pairs and ii) the embedding of the object into a synthetic scene in a pseudo 3D manner. We implement the application using a highly customizable software framework for image processing tasks.
{"title":"Interactive free viewpoint video from multiple stereo","authors":"C. Weigel, S. Schwarz, T. Korn, Martin Wallebohr","doi":"10.1109/3DTV.2009.5069663","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069663","url":null,"abstract":"We present a system for rendering free viewpoint video from data acquired by one or more stereo camera pairs in advance. The free viewpoint video can be observed standalone or shown embedded in a synthetic computer graphics scene. Compared to state-of-the art free viewpoint video applications less cameras are required. The system is scalable in terms of adding more stereo pairs in order to increase the viewing latitude around the object and is therefore adaptable to different kinds of application such as quality assessment tasks or virtual fairs. The main contribution of this paper are i) the scalable extension of the system by additional stereo pairs and ii) the embedding of the object into a synthetic scene in a pseudo 3D manner. We implement the application using a highly customizable software framework for image processing tasks.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127704190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069625
Chenglei Wu, Xun Cao, Qionghai Dai
We present an algorithm that fuses Multi-view stereo (MVS) and photometric stereo to reconstruct 3D model of objects filmed by multiple cameras under varying illuminations. Firstly, we obtain the surface normal scaled by albedo for each view through photometric stereo techniques. Then, based on the scaled normal, a new correspondence matching method, namely surface-consistency metric, is proposed to acquire accurate 3D positions of pixels through triangulation. After filtering the point cloud, a Poisson surface reconstruction is applied to obtain a watertight mesh. The algorithm has been implemented based on our multi-camera and multi-light acquisition system. We validate the method by complete reconstruction of challenging real objects and show experimentally that this technique can greatly improve on previous MVS results.
{"title":"Accurate 3D reconstruction via surface-consistency","authors":"Chenglei Wu, Xun Cao, Qionghai Dai","doi":"10.1109/3DTV.2009.5069625","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069625","url":null,"abstract":"We present an algorithm that fuses Multi-view stereo (MVS) and photometric stereo to reconstruct 3D model of objects filmed by multiple cameras under varying illuminations. Firstly, we obtain the surface normal scaled by albedo for each view through photometric stereo techniques. Then, based on the scaled normal, a new correspondence matching method, namely surface-consistency metric, is proposed to acquire accurate 3D positions of pixels through triangulation. After filtering the point cloud, a Poisson surface reconstruction is applied to obtain a watertight mesh. The algorithm has been implemented based on our multi-camera and multi-light acquisition system. We validate the method by complete reconstruction of challenging real objects and show experimentally that this technique can greatly improve on previous MVS results.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131227687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069630
Feng Lu, Haoqian Wang, Xiangyang Ji, Guihua Er
To save bit-rate in stereo video application, asymmetric view coding is introduced, which encodes the stereo views with different qualities. However, quality assessment on asymmetric view coding is difficult, because the impact of the degraded view upon the 3D percept depends on Human Visual System (HVS) and cannot be indicated by conventional metrics. This paper introduces a quality assessment model based on the observed phenomenon that spatial frequency determines view domination under the action of HVS. A metric is proposed based on this model for assessing the quality of asymmetric view coding. Experimental results are presented to show that the proposed metric provides accordant assessment with the subjective evaluation.
{"title":"Quality assessment of 3D asymmetric view coding using spatial frequency dominance model","authors":"Feng Lu, Haoqian Wang, Xiangyang Ji, Guihua Er","doi":"10.1109/3DTV.2009.5069630","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069630","url":null,"abstract":"To save bit-rate in stereo video application, asymmetric view coding is introduced, which encodes the stereo views with different qualities. However, quality assessment on asymmetric view coding is difficult, because the impact of the degraded view upon the 3D percept depends on Human Visual System (HVS) and cannot be indicated by conventional metrics. This paper introduces a quality assessment model based on the observed phenomenon that spatial frequency determines view domination under the action of HVS. A metric is proposed based on this model for assessing the quality of asymmetric view coding. Experimental results are presented to show that the proposed metric provides accordant assessment with the subjective evaluation.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123223915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069619
Hang Shao, Xun Cao, Guihua Er
In this paper, a novel objective evaluation of depth image based rendering(DIBR) is proposed for the 3D video in format of a monocular video augmented by the gray-scale depth image. The metric is composed of Color and Sharpness of Edge Distortion(CSED) measure. Color distortion measures the luminance loss of the rendered image compared with the reference, and sharpness of edge distortion calculates a depth-weighted proportion of remaining edge to the original edge. Comparing to the conventional quality metrics such as MSE and PSNR, our metric represents not only the color artifact but also the synthesis error with above two aspects. Subjective assessment of the different rendering methods is done as well, and the obtained results show significant agreement with our objective metric.
{"title":"Objective quality assessment of depth image based rendering in 3DTV system","authors":"Hang Shao, Xun Cao, Guihua Er","doi":"10.1109/3DTV.2009.5069619","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069619","url":null,"abstract":"In this paper, a novel objective evaluation of depth image based rendering(DIBR) is proposed for the 3D video in format of a monocular video augmented by the gray-scale depth image. The metric is composed of Color and Sharpness of Edge Distortion(CSED) measure. Color distortion measures the luminance loss of the rendered image compared with the reference, and sharpness of edge distortion calculates a depth-weighted proportion of remaining edge to the original edge. Comparing to the conventional quality metrics such as MSE and PSNR, our metric represents not only the color artifact but also the synthesis error with above two aspects. Subjective assessment of the different rendering methods is done as well, and the obtained results show significant agreement with our objective metric.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"16 6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125835419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, a real-time high-resolution multi-view video transport system which can deliver multi-view video over IP networks is proposed. Video streams are encoded with H.264/AVS. Owing to the massive amount of data involved, multi-view video is delivered in two separate IP channels. Since packets losses always occur in IP networks, a novel packets processing method is employed in the proposed system to hold the correlation between views for loss data recover. Additionally, an error concealment scheme for multi-view stereo video is exploited in this transport system, in order to solve the packet loss problem in IP networks. The experimental results represent that the proposed transport system is feasible for multi-view video in IP networks.
{"title":"Real-time transmission of high-resolution multi-view stereo video over IP networks","authors":"Yuan Zhou, Chunping Hou, Zhigang Jin, Lei Yang, Jiachen Yang, Jichang Guo","doi":"10.1109/3DTV.2009.5069657","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069657","url":null,"abstract":"In this paper, a real-time high-resolution multi-view video transport system which can deliver multi-view video over IP networks is proposed. Video streams are encoded with H.264/AVS. Owing to the massive amount of data involved, multi-view video is delivered in two separate IP channels. Since packets losses always occur in IP networks, a novel packets processing method is employed in the proposed system to hold the correlation between views for loss data recover. Additionally, an error concealment scheme for multi-view stereo video is exploited in this transport system, in order to solve the packet loss problem in IP networks. The experimental results represent that the proposed transport system is feasible for multi-view video in IP networks.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126006777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-04DOI: 10.1109/3DTV.2009.5069662
K. Klimaszewski, K. Wegner, M. Domański
The paper deals with prospective 3D video transmission systems that would use compression of both multiview video and depth maps. The paper addresses the problem of quality of views synthesized from other views transmitted together with depth information. For the state-of-the-art depth map estimation and view synthesize techniques, the paper proves that AVC/SVC-based Multiview Video Coding technique can be used for compression of both view pictures and depth maps. The paper reports extensive experiments where synthesized video quality has been estimated by use of both PSNR index and subjective assessment. Defined is the critical value of depth quantization parameter as a function of the reference view quantization parameter. For smaller depth map quantization parameters, depth map compression has negligible influence on fidelity of synthesized views.
{"title":"Distortions of synthesized views caused by compression of views and depth maps","authors":"K. Klimaszewski, K. Wegner, M. Domański","doi":"10.1109/3DTV.2009.5069662","DOIUrl":"https://doi.org/10.1109/3DTV.2009.5069662","url":null,"abstract":"The paper deals with prospective 3D video transmission systems that would use compression of both multiview video and depth maps. The paper addresses the problem of quality of views synthesized from other views transmitted together with depth information. For the state-of-the-art depth map estimation and view synthesize techniques, the paper proves that AVC/SVC-based Multiview Video Coding technique can be used for compression of both view pictures and depth maps. The paper reports extensive experiments where synthesized video quality has been estimated by use of both PSNR index and subjective assessment. Defined is the critical value of depth quantization parameter as a function of the reference view quantization parameter. For smaller depth map quantization parameters, depth map compression has negligible influence on fidelity of synthesized views.","PeriodicalId":230128,"journal":{"name":"2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132495118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：