Muhammad Shahid Farid, Badi uz Zaman Babar, Muhammad Hassan Khan
{"title":"Efficient representation of disoccluded regions in 3D video coding","authors":"Muhammad Shahid Farid, Badi uz Zaman Babar, Muhammad Hassan Khan","doi":"10.1007/s12243-024-01019-3","DOIUrl":null,"url":null,"abstract":"<p>Three-dimensional (3D) video technology has gained immense admiration in recent times due to its numerous applications, particularly in the television and cinema industry. Three-dimensional television (3DTV) and free-viewpoint television (FTV) are two well-known applications that provide the end-user with a real-world and high-quality 3D display. In both applications, multiple views captured from different viewpoints are rendered simultaneously to offer depth sensation to the viewer. A large number of views are needed to enable FTV. However, transmitting this massive amount of data is challenging due to bandwidth limitations. Multiview video-plus-depth (MVD) is the most popular format where in addition to color images, corresponding depth information is also available which represents the scene geometry. The MVD format with the help of depth image-based rendering (DIBR) enables the generation of views at novel viewpoints. In this paper, we introduce a panorama-based representation of MVD data with an efficient keyframe-based disocclusions handling technique. The panorama view for a stereo pair with depth is constructed from the left view and the novel appearing region of the right view which is not visible from the left viewpoint. The disocclusions that appear in the right view when obtained from the DIBR of the left view are collected in a special frame named as keyframe. On the decoder side, the left view is available with a simple crop of panorama view. The right view is obtained through DIBR of the left view combined with the appearing region from the panorama view. The disocclusions in this warped view are filled from the keyframe. The panorama view with additional keyframes and the corresponding depth map are compressed using the standard HEVC codec. The experimental evaluations performed on standard MVD sequences showed that the proposed scheme achieves excellent video quality while saving considerable bit rate compared to HEVC simulcast.</p>","PeriodicalId":50761,"journal":{"name":"Annals of Telecommunications","volume":null,"pages":null},"PeriodicalIF":1.8000,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Telecommunications","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12243-024-01019-3","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Three-dimensional (3D) video technology has gained immense admiration in recent times due to its numerous applications, particularly in the television and cinema industry. Three-dimensional television (3DTV) and free-viewpoint television (FTV) are two well-known applications that provide the end-user with a real-world and high-quality 3D display. In both applications, multiple views captured from different viewpoints are rendered simultaneously to offer depth sensation to the viewer. A large number of views are needed to enable FTV. However, transmitting this massive amount of data is challenging due to bandwidth limitations. Multiview video-plus-depth (MVD) is the most popular format where in addition to color images, corresponding depth information is also available which represents the scene geometry. The MVD format with the help of depth image-based rendering (DIBR) enables the generation of views at novel viewpoints. In this paper, we introduce a panorama-based representation of MVD data with an efficient keyframe-based disocclusions handling technique. The panorama view for a stereo pair with depth is constructed from the left view and the novel appearing region of the right view which is not visible from the left viewpoint. The disocclusions that appear in the right view when obtained from the DIBR of the left view are collected in a special frame named as keyframe. On the decoder side, the left view is available with a simple crop of panorama view. The right view is obtained through DIBR of the left view combined with the appearing region from the panorama view. The disocclusions in this warped view are filled from the keyframe. The panorama view with additional keyframes and the corresponding depth map are compressed using the standard HEVC codec. The experimental evaluations performed on standard MVD sequences showed that the proposed scheme achieves excellent video quality while saving considerable bit rate compared to HEVC simulcast.
期刊介绍:
Annals of Telecommunications is an international journal publishing original peer-reviewed papers in the field of telecommunications. It covers all the essential branches of modern telecommunications, ranging from digital communications to communication networks and the internet, to software, protocols and services, uses and economics. This large spectrum of topics accounts for the rapid convergence through telecommunications of the underlying technologies in computers, communications, content management towards the emergence of the information and knowledge society. As a consequence, the Journal provides a medium for exchanging research results and technological achievements accomplished by the European and international scientific community from academia and industry.