Efficient representation of disoccluded regions in 3D video coding

IF 1.8 4区 计算机科学 Q3 TELECOMMUNICATIONS Annals of Telecommunications Pub Date : 2024-03-12 DOI:10.1007/s12243-024-01019-3
Muhammad Shahid Farid, Badi uz Zaman Babar, Muhammad Hassan Khan
{"title":"Efficient representation of disoccluded regions in 3D video coding","authors":"Muhammad Shahid Farid, Badi uz Zaman Babar, Muhammad Hassan Khan","doi":"10.1007/s12243-024-01019-3","DOIUrl":null,"url":null,"abstract":"<p>Three-dimensional (3D) video technology has gained immense admiration in recent times due to its numerous applications, particularly in the television and cinema industry. Three-dimensional television (3DTV) and free-viewpoint television (FTV) are two well-known applications that provide the end-user with a real-world and high-quality 3D display. In both applications, multiple views captured from different viewpoints are rendered simultaneously to offer depth sensation to the viewer. A large number of views are needed to enable FTV. However, transmitting this massive amount of data is challenging due to bandwidth limitations. Multiview video-plus-depth (MVD) is the most popular format where in addition to color images, corresponding depth information is also available which represents the scene geometry. The MVD format with the help of depth image-based rendering (DIBR) enables the generation of views at novel viewpoints. In this paper, we introduce a panorama-based representation of MVD data with an efficient keyframe-based disocclusions handling technique. The panorama view for a stereo pair with depth is constructed from the left view and the novel appearing region of the right view which is not visible from the left viewpoint. The disocclusions that appear in the right view when obtained from the DIBR of the left view are collected in a special frame named as keyframe. On the decoder side, the left view is available with a simple crop of panorama view. The right view is obtained through DIBR of the left view combined with the appearing region from the panorama view. The disocclusions in this warped view are filled from the keyframe. The panorama view with additional keyframes and the corresponding depth map are compressed using the standard HEVC codec. The experimental evaluations performed on standard MVD sequences showed that the proposed scheme achieves excellent video quality while saving considerable bit rate compared to HEVC simulcast.</p>","PeriodicalId":50761,"journal":{"name":"Annals of Telecommunications","volume":"4 1","pages":""},"PeriodicalIF":1.8000,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Telecommunications","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12243-024-01019-3","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0

Abstract

Three-dimensional (3D) video technology has gained immense admiration in recent times due to its numerous applications, particularly in the television and cinema industry. Three-dimensional television (3DTV) and free-viewpoint television (FTV) are two well-known applications that provide the end-user with a real-world and high-quality 3D display. In both applications, multiple views captured from different viewpoints are rendered simultaneously to offer depth sensation to the viewer. A large number of views are needed to enable FTV. However, transmitting this massive amount of data is challenging due to bandwidth limitations. Multiview video-plus-depth (MVD) is the most popular format where in addition to color images, corresponding depth information is also available which represents the scene geometry. The MVD format with the help of depth image-based rendering (DIBR) enables the generation of views at novel viewpoints. In this paper, we introduce a panorama-based representation of MVD data with an efficient keyframe-based disocclusions handling technique. The panorama view for a stereo pair with depth is constructed from the left view and the novel appearing region of the right view which is not visible from the left viewpoint. The disocclusions that appear in the right view when obtained from the DIBR of the left view are collected in a special frame named as keyframe. On the decoder side, the left view is available with a simple crop of panorama view. The right view is obtained through DIBR of the left view combined with the appearing region from the panorama view. The disocclusions in this warped view are filled from the keyframe. The panorama view with additional keyframes and the corresponding depth map are compressed using the standard HEVC codec. The experimental evaluations performed on standard MVD sequences showed that the proposed scheme achieves excellent video quality while saving considerable bit rate compared to HEVC simulcast.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
在 3D 视频编码中高效表示不包括的区域
近年来,三维(3D)视频技术因其众多的应用而备受推崇,尤其是在电视和电影行业。三维电视(3DTV)和自由视点电视(FTV)是两种著名的应用,可为终端用户提供真实世界的高质量三维显示。在这两种应用中,从不同视点捕捉的多个视图会同时渲染,为观众提供深度感。FTV 需要大量的视图。然而,由于带宽限制,传输这种海量数据具有挑战性。多视角视频加深度(MVD)是最流行的格式,除了彩色图像外,还提供相应的深度信息,以表示场景的几何形状。MVD 格式在基于深度图像的渲染(DIBR)的帮助下,可以生成新视角的视图。在本文中,我们介绍了一种基于全景的 MVD 数据表示方法,以及一种高效的基于关键帧的不连贯处理技术。有深度的立体对的全景视图是由左视图和右视图中从左视角看不到的新出现区域构建的。从左视图的 DIBR 中获得的右视图中出现的不包含物被收集到一个特殊的帧中,该帧被命名为关键帧。在解码器方面,左视图可通过简单的全景裁剪获得。右视图是通过左视图的 DIBR 和全景图中出现的区域结合获得的。该扭曲视图中的不连贯区域由关键帧填充。带有附加关键帧的全景图和相应的深度图使用标准的 HEVC 编解码器进行压缩。在标准 MVD 序列上进行的实验评估表明,与 HEVC 同步广播相比,所提出的方案在节省大量比特率的同时,还实现了出色的视频质量。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Annals of Telecommunications
Annals of Telecommunications 工程技术-电信学
CiteScore
5.20
自引率
5.30%
发文量
37
审稿时长
4.5 months
期刊介绍: Annals of Telecommunications is an international journal publishing original peer-reviewed papers in the field of telecommunications. It covers all the essential branches of modern telecommunications, ranging from digital communications to communication networks and the internet, to software, protocols and services, uses and economics. This large spectrum of topics accounts for the rapid convergence through telecommunications of the underlying technologies in computers, communications, content management towards the emergence of the information and knowledge society. As a consequence, the Journal provides a medium for exchanging research results and technological achievements accomplished by the European and international scientific community from academia and industry.
期刊最新文献
Editorial of 6GNet 2023 special issue On the (in)efficiency of fuzzing network protocols Investigation of LDPC codes with interleaving for 5G wireless networks Opportunistic data gathering in IoT networks using an energy-efficient data aggregation mechanism Joint MEC selection and wireless resource allocation in 5G RAN
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1