Elevating traditional 2-dimensional (2D) plane display to 4-dimensional (4D) light field display can significantly enhance users’ immersion and realism, because light field image (LFI) provides various visual cues in terms of multi-view disparity, motion disparity, and selective focus. Therefore, it is crucial to establish a light field image quality assessment (LF-IQA) model that aligns with human visual perception characteristics. However, it has always been a challenge to evaluate the perceptual quality of multiple light field visual cues simultaneously and consistently. To this end, this paper proposes a Transformer-based explicit learning of light field geometry for the no-reference light field image quality assessment. Specifically, to explicitly learn the light field epipolar geometry, we stack up light field sub-aperture images (SAIs) to form four SAI stacks according to four specific light field angular directions, and use a sub-grouping strategy to hierarchically learn the local and global light field geometric features. Then, a Transformer encoder with a spatial-shift tokenization strategy is applied to learn structure-aware light field geometric distortion representation, which is used to regress the final quality score. Evaluation experiments are carried out on three commonly used light field image quality assessment datasets: Win5-LID, NBU-LF1.0, and MPI-LFA. Experimental results demonstrate that our model outperforms state-of-the-art methods and exhibits a high correlation with human perception. The source code is publicly available at https://github.com/windyz77/GeoNRLFIQA .
将传统的二维（2D）平面显示提升到四维（4D）光场显示，可以显著增强用户的沉浸感和真实感，因为光场图像（LFI）提供了多视角差异、运动差异和选择性聚焦等多种视觉线索。因此，建立一个符合人类视觉感知特征的光场图像质量评估（LF-IQA）模型至关重要。然而，如何同时、一致地评估多个光场视觉线索的感知质量一直是个难题。为此，本文提出了一种基于变换器的光场几何显式学习方法，用于无参照光场图像质量评估。具体来说，为了显式学习光场外极几何，我们将光场子孔径图像（SAI）按照四个特定的光场角度方向堆叠成四个 SAI 堆栈，并使用子分组策略分层学习局部和全局光场几何特征。然后，采用空间偏移标记化策略的变换器编码器学习结构感知光场几何失真表示，并以此回归最终质量得分。评估实验在三个常用的光场图像质量评估数据集上进行：Win5-LID、NBU-LF1.0 和 MPI-LFA。实验结果表明，我们的模型优于最先进的方法，并且与人类感知具有很高的相关性。源代码可通过 https://github.com/windyz77/GeoNRLFIQA 公开获取。
{"title":"Transformer-Based Light Field Geometry Learning for No-Reference Light Field Image Quality Assessment","authors":"Lili Lin;Siyu Bai;Mengjia Qu;Xuehui Wei;Luyao Wang;Feifan Wu;Biao Liu;Wenhui Zhou;Ercan Engin Kuruoglu","doi":"10.1109/TBC.2024.3353579","DOIUrl":"10.1109/TBC.2024.3353579","url":null,"abstract":"Elevating traditional 2-dimensional (2D) plane display to 4-dimensional (4D) light field display can significantly enhance users’ immersion and realism, because light field image (LFI) provides various visual cues in terms of multi-view disparity, motion disparity, and selective focus. Therefore, it is crucial to establish a light field image quality assessment (LF-IQA) model that aligns with human visual perception characteristics. However, it has always been a challenge to evaluate the perceptual quality of multiple light field visual cues simultaneously and consistently. To this end, this paper proposes a Transformer-based explicit learning of light field geometry for the no-reference light field image quality assessment. Specifically, to explicitly learn the light field epipolar geometry, we stack up light field sub-aperture images (SAIs) to form four SAI stacks according to four specific light field angular directions, and use a sub-grouping strategy to hierarchically learn the local and global light field geometric features. Then, a Transformer encoder with a spatial-shift tokenization strategy is applied to learn structure-aware light field geometric distortion representation, which is used to regress the final quality score. Evaluation experiments are carried out on three commonly used light field image quality assessment datasets: Win5-LID, NBU-LF1.0, and MPI-LFA. Experimental results demonstrate that our model outperforms state-of-the-art methods and exhibits a high correlation with human perception. The source code is publicly available at \u0000<uri>https://github.com/windyz77/GeoNRLFIQA</uri>\u0000.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"597-606"},"PeriodicalIF":4.5,"publicationDate":"2024-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Inter-tower communications networks (ITCN) and wireless in-band distribution links (IDL) reuse the same broadcast spectrum for establishing communications links between transmitter towers and for wireless backhaul by multiplexing the ITCN/IDL signals with the broadcast signal into a frame for transmission. In single-frequency networks (SFN) environment, where all the transmitter towers broadcast the same preamble and in-band pilots for improving TV coverage and received signal strength, receiving the desired ITCN/IDL signal from a specific transmitter is challenging with the conventional channel estimation techniques. In the Advanced Television Standard Committee (ATSC) 3.0, one unique transmitter identification (TxID) sequence, a spread sequence overlaid with the preamble signal, is assigned for each transmitter for the purpose of SFN planning and synchronization. By using the TxID sequence, the channel estimation of a specific transmitter becomes feasible. However, the accuracy of existing TxID-based channel estimation is limited due to interferences from the preamble signal and the co-channel TxIDs, as well as the non-orthogonality of the TxID sequence. Several high-accuracy channel estimation schemes based on the TxID sequence are proposed in this paper, which enable IDL and ITCN with very high data rate transmission, e.g., 1024 QAM modulation.
{"title":"High Accuracy Channel Estimation With TxID Sequence in ATSC 3.0 SFN","authors":"Zhihong Hunter Hong;Yiyan Wu;Wei Li;Liang Zhang;Zhiwen Zhu;Sung-Ik Park;Namho Hur;Eneko Iradier;Jon Montalban","doi":"10.1109/TBC.2024.3353577","DOIUrl":"10.1109/TBC.2024.3353577","url":null,"abstract":"Inter-tower communications networks (ITCN) and wireless in-band distribution links (IDL) reuse the same broadcast spectrum for establishing communications links between transmitter towers and for wireless backhaul by multiplexing the ITCN/IDL signals with the broadcast signal into a frame for transmission. In single-frequency networks (SFN) environment, where all the transmitter towers broadcast the same preamble and in-band pilots for improving TV coverage and received signal strength, receiving the desired ITCN/IDL signal from a specific transmitter is challenging with the conventional channel estimation techniques. In the Advanced Television Standard Committee (ATSC) 3.0, one unique transmitter identification (TxID) sequence, a spread sequence overlaid with the preamble signal, is assigned for each transmitter for the purpose of SFN planning and synchronization. By using the TxID sequence, the channel estimation of a specific transmitter becomes feasible. However, the accuracy of existing TxID-based channel estimation is limited due to interferences from the preamble signal and the co-channel TxIDs, as well as the non-orthogonality of the TxID sequence. Several high-accuracy channel estimation schemes based on the TxID sequence are proposed in this paper, which enable IDL and ITCN with very high data rate transmission, e.g., 1024 QAM modulation.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"391-400"},"PeriodicalIF":4.5,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947447","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-30DOI: 10.1109/TBC.2024.3353568
Linyao Gao;Zhu Li;Lizhi Hou;Yiling Xu;Jun Sun
Video-based point cloud compression (V-PCC) has achieved remarkable compression efficiency, which converts point clouds into videos and leverages video codecs for coding. For lossy compression, the undesirable artifacts of attribute images always degrade the point clouds attribute reconstruction quality. In this paper, we propose an Occupancy-assisted Compression Artifact Removal Network (OCARNet) to remove the distortions of V-PCC decoded attribute images for high-quality point cloud attribute reconstruction. Specifically, the occupancy information is fed into network as a prior knowledge to provide more spatial and structural information and to assist in eliminating the distortions of the texture regions. To aggregate the occupancy information effectively, we design a multi-level feature fusion framework with Channel-Spatial Attention based Residual Blocks (CSARB), where the short and long residual connections are jointly employed to capture the local context and long-range dependency. Besides, we propose a Masked Mean Square Error (MMSE) loss function based on the occupancy information to train our proposed network to focus on estimating the attribute artifacts of the occupied regions. To the best of our knowledge, this is the first learning-based attribute artifact removal method for V-PCC. Experimental results demonstrate that our framework outperforms existing state-of-the-art methods and shows the effectiveness on both objective and subjective quality comparisons.
{"title":"Occupancy-Assisted Attribute Artifact Reduction for Video-Based Point Cloud Compression","authors":"Linyao Gao;Zhu Li;Lizhi Hou;Yiling Xu;Jun Sun","doi":"10.1109/TBC.2024.3353568","DOIUrl":"10.1109/TBC.2024.3353568","url":null,"abstract":"Video-based point cloud compression (V-PCC) has achieved remarkable compression efficiency, which converts point clouds into videos and leverages video codecs for coding. For lossy compression, the undesirable artifacts of attribute images always degrade the point clouds attribute reconstruction quality. In this paper, we propose an Occupancy-assisted Compression Artifact Removal Network (OCARNet) to remove the distortions of V-PCC decoded attribute images for high-quality point cloud attribute reconstruction. Specifically, the occupancy information is fed into network as a prior knowledge to provide more spatial and structural information and to assist in eliminating the distortions of the texture regions. To aggregate the occupancy information effectively, we design a multi-level feature fusion framework with Channel-Spatial Attention based Residual Blocks (CSARB), where the short and long residual connections are jointly employed to capture the local context and long-range dependency. Besides, we propose a Masked Mean Square Error (MMSE) loss function based on the occupancy information to train our proposed network to focus on estimating the attribute artifacts of the occupied regions. To the best of our knowledge, this is the first learning-based attribute artifact removal method for V-PCC. Experimental results demonstrate that our framework outperforms existing state-of-the-art methods and shows the effectiveness on both objective and subjective quality comparisons.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"667-680"},"PeriodicalIF":4.5,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-25DOI: 10.1109/TBC.2024.3349768
Sujie Shao;Linlin Zhang;Fei Qi
With the improvement of terminal intelligence and the enrichment of digital content, terminal density is showing an explosive growth trend, and the traffic carried by IPTV and other services is rapidly increasing. HDHB WLAN (High-Density High-Bandwidth Wireless LAN) is becoming a dominant form of wireless LAN. However, the RSSI-based access mode has led to a notable load imbalance, and the resource competition mode based on random access intensifies the difficulty of access resource acquisition, which exacerbates the traffic challenges faced by WLAN. IEEE 802.11ax somewhat alleviates traffic pressure, but it does not fundamentally solve these problems. This paper introduces an access optimization mechanism for the 802.11ax HDHB WLAN, which aims to achieve load balancing while considering competition avoidance, alleviating the pressure of IPTV traffic. First, an 802.11ax access optimization architecture for HDHB WLAN is constructed, aimed at alleviating traffic pressure and meeting the quality requirements of IPTV and other services by modifying access processes of terminals. Next, a terminal information acquisition and interactive access control strategy based on the trigger frame is devised to obtain accurate parameter information and facilitate orderly concurrent access control for high-density terminals. Additionally, a load balancing and competition avoidance oriented access control method for HDHB WLAN is proposed, including an access optimization control model, and an access strategy generation algorithm based on the Improved DQN algorithm. Finally, simulation results show that the global throughput and load balancing of HDHB WLAN are improved, consequently reducing overall WLAN traffic pressure.
{"title":"Access Optimization in 802.11ax WLAN for Load Balancing and Competition Avoidance of IPTV Traffic","authors":"Sujie Shao;Linlin Zhang;Fei Qi","doi":"10.1109/TBC.2024.3349768","DOIUrl":"10.1109/TBC.2024.3349768","url":null,"abstract":"With the improvement of terminal intelligence and the enrichment of digital content, terminal density is showing an explosive growth trend, and the traffic carried by IPTV and other services is rapidly increasing. HDHB WLAN (High-Density High-Bandwidth Wireless LAN) is becoming a dominant form of wireless LAN. However, the RSSI-based access mode has led to a notable load imbalance, and the resource competition mode based on random access intensifies the difficulty of access resource acquisition, which exacerbates the traffic challenges faced by WLAN. IEEE 802.11ax somewhat alleviates traffic pressure, but it does not fundamentally solve these problems. This paper introduces an access optimization mechanism for the 802.11ax HDHB WLAN, which aims to achieve load balancing while considering competition avoidance, alleviating the pressure of IPTV traffic. First, an 802.11ax access optimization architecture for HDHB WLAN is constructed, aimed at alleviating traffic pressure and meeting the quality requirements of IPTV and other services by modifying access processes of terminals. Next, a terminal information acquisition and interactive access control strategy based on the trigger frame is devised to obtain accurate parameter information and facilitate orderly concurrent access control for high-density terminals. Additionally, a load balancing and competition avoidance oriented access control method for HDHB WLAN is proposed, including an access optimization control model, and an access strategy generation algorithm based on the Improved DQN algorithm. Finally, simulation results show that the global throughput and load balancing of HDHB WLAN are improved, consequently reducing overall WLAN traffic pressure.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"401-412"},"PeriodicalIF":4.5,"publicationDate":"2024-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-25DOI: 10.1109/TBC.2024.3349773
Alireza Esmaeilzehi;Yang Ou;M. Omair Ahmad;M. N. S. Swamy
Enhancing the quality of the images acquired under the water environments is crucial in many broadcast technologies. As the richness of the features generated by deep underwater image enhancement networks improves, the visual signals with higher qualities can be yielded. In view of this, in this paper, we propose a new deep network for the task of underwater image enhancement, in which the network feature generation process is guided by the prior information obtained from various underwater medium transmission map and atmospheric light estimation methods. Further, in order to obtain high values for different image quality assessment metrics associated with the images produced by the proposed network, we introduce a multi-stage training process for our network. In the first stage, the proposed network is trained with the conventional supervised learning technique, whereas, in the second stage, the training process of the network is carried out by the adversarial learning technique. Finally, in the third stage, the training of the network obtained by the conventional supervised learning is continued by the guidance of the one trained by the adversarial learning technique. In the development of the adversarial learning-based stage of our network, we propose a novel multi-discriminator generative adversarial network, which is able to produce images with more realistic textures and structures. The proposed multi-discriminator generative adversarial network employs the discrimination process between the real and fake data in various underwater environment color spaces. The results of different experimentations show the effectiveness of the proposed scheme in restoring the high-quality images compared to the other state-of-the-art deep underwater image enhancement networks.
{"title":"DMML: Deep Multi-Prior and Multi-Discriminator Learning for Underwater Image Enhancement","authors":"Alireza Esmaeilzehi;Yang Ou;M. Omair Ahmad;M. N. S. Swamy","doi":"10.1109/TBC.2024.3349773","DOIUrl":"10.1109/TBC.2024.3349773","url":null,"abstract":"Enhancing the quality of the images acquired under the water environments is crucial in many broadcast technologies. As the richness of the features generated by deep underwater image enhancement networks improves, the visual signals with higher qualities can be yielded. In view of this, in this paper, we propose a new deep network for the task of underwater image enhancement, in which the network feature generation process is guided by the prior information obtained from various underwater medium transmission map and atmospheric light estimation methods. Further, in order to obtain high values for different image quality assessment metrics associated with the images produced by the proposed network, we introduce a multi-stage training process for our network. In the first stage, the proposed network is trained with the conventional supervised learning technique, whereas, in the second stage, the training process of the network is carried out by the adversarial learning technique. Finally, in the third stage, the training of the network obtained by the conventional supervised learning is continued by the guidance of the one trained by the adversarial learning technique. In the development of the adversarial learning-based stage of our network, we propose a novel multi-discriminator generative adversarial network, which is able to produce images with more realistic textures and structures. The proposed multi-discriminator generative adversarial network employs the discrimination process between the real and fake data in various underwater environment color spaces. The results of different experimentations show the effectiveness of the proposed scheme in restoring the high-quality images compared to the other state-of-the-art deep underwater image enhancement networks.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"637-653"},"PeriodicalIF":4.5,"publicationDate":"2024-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947564","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-22DOI: 10.1109/TBC.2024.3349790
Zhitong He;Kewu Peng;Chao Zhang;Jian Song
Digital terrestrial television multimedia broadcasting-advanced (DTMB-A) proposed by China is served as a 2nd generation digital terrestrial television broadcasting (DTTB) standard with advanced forward error correction coding schemes. Nevertheless, to adapt low signal-to-noise ratio (SNR) scenarios such as in cloud transmission systems, LDPC codes with low rates are required for DTMB-A. In this paper, the new design of low-rate DTMB-A LDPC codes is presented systematically. Specifically, a rate-compatible Raptor-Like structure of low-rate DTMB-A LDPC codes is presented, which supports multiple low code rates with constant code length. Then a new construction method is proposed for low-rate DTMB-A LDPC codes, where progressive block extension is employed and the minimum distance is majorly optimized such that the minimum distance increases after each block extension. Finally, the performance of the constructed DTMB-A LDPC codes with two low code rates of 1/3 and 1/4 are simulated and compared with ATSC 3.0 LDPC codes, which demonstrates the effectiveness of our design.
{"title":"Low-Rate LDPC Code Design for DTMB-A","authors":"Zhitong He;Kewu Peng;Chao Zhang;Jian Song","doi":"10.1109/TBC.2024.3349790","DOIUrl":"10.1109/TBC.2024.3349790","url":null,"abstract":"Digital terrestrial television multimedia broadcasting-advanced (DTMB-A) proposed by China is served as a 2nd generation digital terrestrial television broadcasting (DTTB) standard with advanced forward error correction coding schemes. Nevertheless, to adapt low signal-to-noise ratio (SNR) scenarios such as in cloud transmission systems, LDPC codes with low rates are required for DTMB-A. In this paper, the new design of low-rate DTMB-A LDPC codes is presented systematically. Specifically, a rate-compatible Raptor-Like structure of low-rate DTMB-A LDPC codes is presented, which supports multiple low code rates with constant code length. Then a new construction method is proposed for low-rate DTMB-A LDPC codes, where progressive block extension is employed and the minimum distance is majorly optimized such that the minimum distance increases after each block extension. Finally, the performance of the constructed DTMB-A LDPC codes with two low code rates of 1/3 and 1/4 are simulated and compared with ATSC 3.0 LDPC codes, which demonstrates the effectiveness of our design.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"739-746"},"PeriodicalIF":4.5,"publicationDate":"2024-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent advancements in SDRTV-to-HDRTV conversion have yielded impressive results in reconstructing high dynamic range television (HDRTV) videos from standard dynamic range television (SDRTV) videos. However, the practical applications of these techniques are limited for ultra-high definition (UHD) video systems due to their high computational and memory costs. In this paper, we propose EffiHDR, an efficient framework primarily operating in the downsampled space, effectively reducing the computational and memory demands. Our framework comprises a real-time SDRTV-to-HDRTV Reconstruction model and a plug-and-play HDRTV Enhancement model. The SDRTV-to-HDRTV Reconstruction model learns affine transformation coefficients instead of directly predicting output pixels to preserve high-frequency information and mitigate information loss caused by downsampling. It decomposes SDRTV-to-HDR mapping into pixel intensity-dependent and local-dependent affine transformations. The pixel intensity-dependent transformation leverages global contexts and pixel intensity conditions to transform SDRTV pixels to the HDRTV domain. The local-dependent transformation predicts affine coefficients based on local contexts, further enhancing dynamic range, local contrast, and color tone. Additionally, we introduce a plug-and-play HDRTV Enhancement model based on an efficient Transformer-based U-net, which enhances luminance and color details in challenging recovery scenarios. Experimental results demonstrate that our SDRTV-to-HDRTV Reconstruction model achieves real-time 4K conversion with impressive performance. When combined with the HDRTV Enhancement model, our approach outperforms state-of-the-art methods in performance and efficiency.
{"title":"EffiHDR: An Efficient Framework for HDRTV Reconstruction and Enhancement in UHD Systems","authors":"Hengsheng Zhang;Xueyi Zou;Guo Lu;Li Chen;Li Song;Wenjun Zhang","doi":"10.1109/TBC.2023.3345657","DOIUrl":"10.1109/TBC.2023.3345657","url":null,"abstract":"Recent advancements in SDRTV-to-HDRTV conversion have yielded impressive results in reconstructing high dynamic range television (HDRTV) videos from standard dynamic range television (SDRTV) videos. However, the practical applications of these techniques are limited for ultra-high definition (UHD) video systems due to their high computational and memory costs. In this paper, we propose EffiHDR, an efficient framework primarily operating in the downsampled space, effectively reducing the computational and memory demands. Our framework comprises a real-time SDRTV-to-HDRTV Reconstruction model and a plug-and-play HDRTV Enhancement model. The SDRTV-to-HDRTV Reconstruction model learns affine transformation coefficients instead of directly predicting output pixels to preserve high-frequency information and mitigate information loss caused by downsampling. It decomposes SDRTV-to-HDR mapping into pixel intensity-dependent and local-dependent affine transformations. The pixel intensity-dependent transformation leverages global contexts and pixel intensity conditions to transform SDRTV pixels to the HDRTV domain. The local-dependent transformation predicts affine coefficients based on local contexts, further enhancing dynamic range, local contrast, and color tone. Additionally, we introduce a plug-and-play HDRTV Enhancement model based on an efficient Transformer-based U-net, which enhances luminance and color details in challenging recovery scenarios. Experimental results demonstrate that our SDRTV-to-HDRTV Reconstruction model achieves real-time 4K conversion with impressive performance. When combined with the HDRTV Enhancement model, our approach outperforms state-of-the-art methods in performance and efficiency.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"620-636"},"PeriodicalIF":4.5,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947566","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-10DOI: 10.1109/TBC.2023.3345646
Wei Zhang;Yunpeng Jing;Yuan Zhang;Tao Lin;Jinyao Yan
UHD live video streaming, with its high video resolution, offers a wealth of fine-grained scene details, presenting opportunities for intricate video analytics. However, current real-time video streaming analytics solutions are inadequate in analyzing these detailed features, often leading to low accuracy in the analysis of small objects with fine details. Furthermore, due to the high bitrate and precision of UHD streaming, existing real-time inference frameworks typically suffer from low analyzed frame rate caused by the significant computational cost involved. To meet the accuracy requirement and improve the analyzed frame rate, we introduce Retina-U, a real-time analytics framework for UHD video streaming. Specifically, we first present SECT, a real-time DNN model level inference model to enhance inference accuracy in dynamic UHD streaming with an abundance of small objects. SECT uses a slicing-based enhanced inference (SEI) method and Cascade Sparse Queries (CSQ) based-fine tuning to improve the accuracy, and leverages a lightweight tracker to achieve high analyzed frame rate. At the system level, to further improve the inference accuracy and bolster the analyzed frame rate, we propose a deep reinforcement learning-based resource management algorithm for real-time joint network adaptation, resource allocation, and server selection. By simultaneously considering the network and computational resources, we can maximize the comprehensive analytic performance in a dynamic and complex environment. Experimental results demonstrate the effectiveness of Retina-U, showcasing improvements in accuracy of up to 38.01% and inference speed acceleration of up to 24.33%.
{"title":"Retina-U: A Two-Level Real-Time Analytics Framework for UHD Live Video Streaming","authors":"Wei Zhang;Yunpeng Jing;Yuan Zhang;Tao Lin;Jinyao Yan","doi":"10.1109/TBC.2023.3345646","DOIUrl":"10.1109/TBC.2023.3345646","url":null,"abstract":"UHD live video streaming, with its high video resolution, offers a wealth of fine-grained scene details, presenting opportunities for intricate video analytics. However, current real-time video streaming analytics solutions are inadequate in analyzing these detailed features, often leading to low accuracy in the analysis of small objects with fine details. Furthermore, due to the high bitrate and precision of UHD streaming, existing real-time inference frameworks typically suffer from low analyzed frame rate caused by the significant computational cost involved. To meet the accuracy requirement and improve the analyzed frame rate, we introduce Retina-U, a real-time analytics framework for UHD video streaming. Specifically, we first present SECT, a real-time DNN model level inference model to enhance inference accuracy in dynamic UHD streaming with an abundance of small objects. SECT uses a slicing-based enhanced inference (SEI) method and Cascade Sparse Queries (CSQ) based-fine tuning to improve the accuracy, and leverages a lightweight tracker to achieve high analyzed frame rate. At the system level, to further improve the inference accuracy and bolster the analyzed frame rate, we propose a deep reinforcement learning-based resource management algorithm for real-time joint network adaptation, resource allocation, and server selection. By simultaneously considering the network and computational resources, we can maximize the comprehensive analytic performance in a dynamic and complex environment. Experimental results demonstrate the effectiveness of Retina-U, showcasing improvements in accuracy of up to 38.01% and inference speed acceleration of up to 24.33%.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"429-440"},"PeriodicalIF":4.5,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-10DOI: 10.1109/TBC.2023.3340042
Liping Zhao;Zhuge Yan;Zehao Wang;Xu Wang;Keli Hu;Huawen Liu;Tao Lin
During and following the global COVID-19 pandemic, the use of screen content coding applications such as large-scale cloud office, online teaching, and teleconferencing has surged. The vast amount of online data generated by these applications, especially online teaching, has become a vital source of Internet video traffic. Consequently, there is an urgent need for low-complexity online teaching screen content (OTSC) coding techniques. Energy-efficient low-complexity green coding techniques for OTSC, named GCOTSC, are proposed based on the unique characteristics of OTSC. In the inter-frame prediction mode, the input frames are first divided into visually constant frames (VCFs) and non-VCFs using a VCF identifier. A new VCF mode has been proposed to code VCFs efficiently. In the intra-frame prediction mode, a heuristic multi-type least probable option skip mode based on static and dynamic historical information is proposed. Compared with the AVS3 screen content coding algorithm, using the typical online teaching screen content and AVS3 SCC common test condition, the experimental results show that the GOTSC achieves an average 59.06% reduction of encoding complexity in low delay configuration, with almost no impact on coding efficiency.
{"title":"GCOTSC: Green Coding Techniques for Online Teaching Screen Content Implemented in AVS3","authors":"Liping Zhao;Zhuge Yan;Zehao Wang;Xu Wang;Keli Hu;Huawen Liu;Tao Lin","doi":"10.1109/TBC.2023.3340042","DOIUrl":"10.1109/TBC.2023.3340042","url":null,"abstract":"During and following the global COVID-19 pandemic, the use of screen content coding applications such as large-scale cloud office, online teaching, and teleconferencing has surged. The vast amount of online data generated by these applications, especially online teaching, has become a vital source of Internet video traffic. Consequently, there is an urgent need for low-complexity online teaching screen content (OTSC) coding techniques. Energy-efficient low-complexity green coding techniques for OTSC, named GCOTSC, are proposed based on the unique characteristics of OTSC. In the inter-frame prediction mode, the input frames are first divided into visually constant frames (VCFs) and non-VCFs using a VCF identifier. A new VCF mode has been proposed to code VCFs efficiently. In the intra-frame prediction mode, a heuristic multi-type least probable option skip mode based on static and dynamic historical information is proposed. Compared with the AVS3 screen content coding algorithm, using the typical online teaching screen content and AVS3 SCC common test condition, the experimental results show that the GOTSC achieves an average 59.06% reduction of encoding complexity in low delay configuration, with almost no impact on coding efficiency.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 1","pages":"174-182"},"PeriodicalIF":4.5,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-05DOI: 10.1109/TBC.2023.3345642
He Sun;Emanuele Viterbo;Bin Dai;Rongke Liu
The rapid revolution of mobile communication technology provides a great avenue for efficient information transmission to facilitate digital multimedia services. In current 5G systems, broadcasting technology is used to improve the efficiency of information transmission, and polar codes are adopted to improve data transmission reliability. Reducing the decoding latency of polar codes is of great importance for ultra-low-latency and reliable data transmission for 5G broadcasting, which still remains a challenge in digital broadcasting services. In this paper, we propose an aggregation method to construct constituent codes for reducing the decoding latency of polar codes. The aggregation method jointly exploits the structure and reliability of constituent codes to increase the lengths of constituent codes that can be decoded in parallel, thus significantly reducing the decoding latency. Furthermore, an efficient parallel decoding algorithm is integrated with the proposed aggregation method to efficiently decode the reliable constituent codes without sacrificing error-correction performance. Simulation results show that the proposed method significantly reduces the decoding latency as compared to the existing state-of-the-art schemes.
{"title":"Fast Decoding of Polar Codes for Digital Broadcasting Services in 5G","authors":"He Sun;Emanuele Viterbo;Bin Dai;Rongke Liu","doi":"10.1109/TBC.2023.3345642","DOIUrl":"10.1109/TBC.2023.3345642","url":null,"abstract":"The rapid revolution of mobile communication technology provides a great avenue for efficient information transmission to facilitate digital multimedia services. In current 5G systems, broadcasting technology is used to improve the efficiency of information transmission, and polar codes are adopted to improve data transmission reliability. Reducing the decoding latency of polar codes is of great importance for ultra-low-latency and reliable data transmission for 5G broadcasting, which still remains a challenge in digital broadcasting services. In this paper, we propose an aggregation method to construct constituent codes for reducing the decoding latency of polar codes. The aggregation method jointly exploits the structure and reliability of constituent codes to increase the lengths of constituent codes that can be decoded in parallel, thus significantly reducing the decoding latency. Furthermore, an efficient parallel decoding algorithm is integrated with the proposed aggregation method to efficiently decode the reliable constituent codes without sacrificing error-correction performance. Simulation results show that the proposed method significantly reduces the decoding latency as compared to the existing state-of-the-art schemes.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 2","pages":"731-738"},"PeriodicalIF":4.5,"publicationDate":"2024-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139947430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：