Existing no reference image quality assessment(NR-IQA) methods have not incorporated image semantics explicitly in the assessment process, thus overlooking the significant correlation between image content and its quality. To address this gap, we leverages image semantics as guiding information for quality assessment, integrating it explicitly into the NR-IQA process through a Semantic-Guided NR-IQA model(SGIQA), which is based on the Swin Transformer. Specifically, we introduce a Semantic Attention Module and a Perceptual Rule Learning Module. The Semantic Attention Module refines the features extracted by the deep network according to the image content, allowing the network to dynamically extract quality perceptual features according to the semantic context of the image. The Perceptual Rule Learning Module generates parameters for the image quality regression module tailored to the image content, facilitating a dynamic assessment of image quality based on its semantic information. Employing the Swin Transformer and integrating these two modules, we have developed the final semantic-guided NR-IQA model. Extensive experiments on five widely-used IQA datasets demonstrate that our method not only exhibits excellent generalization capabilities but also achieves state-of-the-art performance.
{"title":"SGIQA: Semantic-Guided No-Reference Image Quality Assessment","authors":"Linpeng Pan;Xiaozhe Zhang;Fengying Xie;Haopeng Zhang;Yushan Zheng","doi":"10.1109/TBC.2024.3450320","DOIUrl":"10.1109/TBC.2024.3450320","url":null,"abstract":"Existing no reference image quality assessment(NR-IQA) methods have not incorporated image semantics explicitly in the assessment process, thus overlooking the significant correlation between image content and its quality. To address this gap, we leverages image semantics as guiding information for quality assessment, integrating it explicitly into the NR-IQA process through a Semantic-Guided NR-IQA model(SGIQA), which is based on the Swin Transformer. Specifically, we introduce a Semantic Attention Module and a Perceptual Rule Learning Module. The Semantic Attention Module refines the features extracted by the deep network according to the image content, allowing the network to dynamically extract quality perceptual features according to the semantic context of the image. The Perceptual Rule Learning Module generates parameters for the image quality regression module tailored to the image content, facilitating a dynamic assessment of image quality based on its semantic information. Employing the Swin Transformer and integrating these two modules, we have developed the final semantic-guided NR-IQA model. Extensive experiments on five widely-used IQA datasets demonstrate that our method not only exhibits excellent generalization capabilities but also achieves state-of-the-art performance.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 4","pages":"1292-1301"},"PeriodicalIF":3.2,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Learning-based point cloud compression has achieved great success in Rate-Distortion (RD) efficiency. Existing methods usually utilize Variational AutoEncoder (VAE) network, which might lead to poor detail reconstruction and high computational complexity. To address these issues, we propose a Scale-adaptive Asymmetric Sparse Variational AutoEncoder (SAS-VAE) in this work. First, we develop an Asymmetric Multiscale Sparse Convolution (AMSC), which exploits multi-resolution branches to aggregate multiscale features at encoder, and excludes symmetric feature fusion branches to control the model complexity at decoder. Second, we design a Scale Adaptive Feature Refinement Structure (SAFRS) to adaptively adjust the number of Feature Refinement Modules (FRMs), thereby improving RD performance with an acceptable computational overhead. Third, we implement our framework with AMSC and SAFRS, and train it with an RD loss based on Fine-grained Weighted Binary Cross-Entropy (FWBCE) function. Experimental results on 8iVFB, Owlii, and MVUV datasets show that our method outperforms several popular methods, with a 90.0% time reduction and a 51.8% BD-BR saving compared with V-PCC. The code will be available soon at https://github.com/fancj2017/SAS-VAE .
{"title":"Scale-Adaptive Asymmetric Sparse Variational AutoEncoder for Point Cloud Compression","authors":"Jian Chen;Yingtao Zhu;Wei Huang;Chengdong Lan;Tiesong Zhao","doi":"10.1109/TBC.2024.3437161","DOIUrl":"10.1109/TBC.2024.3437161","url":null,"abstract":"Learning-based point cloud compression has achieved great success in Rate-Distortion (RD) efficiency. Existing methods usually utilize Variational AutoEncoder (VAE) network, which might lead to poor detail reconstruction and high computational complexity. To address these issues, we propose a Scale-adaptive Asymmetric Sparse Variational AutoEncoder (SAS-VAE) in this work. First, we develop an Asymmetric Multiscale Sparse Convolution (AMSC), which exploits multi-resolution branches to aggregate multiscale features at encoder, and excludes symmetric feature fusion branches to control the model complexity at decoder. Second, we design a Scale Adaptive Feature Refinement Structure (SAFRS) to adaptively adjust the number of Feature Refinement Modules (FRMs), thereby improving RD performance with an acceptable computational overhead. Third, we implement our framework with AMSC and SAFRS, and train it with an RD loss based on Fine-grained Weighted Binary Cross-Entropy (FWBCE) function. Experimental results on 8iVFB, Owlii, and MVUV datasets show that our method outperforms several popular methods, with a 90.0% time reduction and a 51.8% BD-BR saving compared with V-PCC. The code will be available soon at \u0000<uri>https://github.com/fancj2017/SAS-VAE</uri>\u0000.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 3","pages":"884-894"},"PeriodicalIF":3.2,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207649","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Retouched Face Image Quality Assessment Based on Differential Perception and Textual Prompt","authors":"Tianwei Zhou, Songbai Tan, Gang Li, Shishun Tian, Chang Tang, Zhihua Wang, Guanghui Yue","doi":"10.1109/tbc.2024.3447454","DOIUrl":"https://doi.org/10.1109/tbc.2024.3447454","url":null,"abstract":"","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"31 1","pages":""},"PeriodicalIF":4.5,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, numerous learned video compression (LVC) methods have emerged, demonstrating rapid developments and satisfactory performance. However, in most previous methods, only the previous one frame is used as reference. Although some works introduce the usage of the previous multiple frames, the exploitation of temporal information is not comprehensive. Our proposed method not only utilizes the short-term information from multiple neighboring frames but also introduces long-term feature information as the reference, which effectively enhances the quality of the context and improves the compression efficiency. In our scheme, we propose the long-term information exploitation mechanism to capture long-term temporal relevance. The update and propagation of long-term information establish an implicit connection between the latent representation of the current frame and distant reference frames, aiding in the generation of long-term context. Meanwhile, the short-term neighboring frames are also utilized to extract local information and generate short-term context. The fusion of long-term context and short-term context results in a more comprehensive and high-quality context to achieve sufficient temporal information mining. Besides, the multiple frames information also helps to improve the efficiency of motion compression. They are used to generate the predicted motion and remove spatio-temporal redundancies in motion information by second-order motion prediction and fusion. Experimental results demonstrate that our proposed efficient learned video compression scheme can achieve 4.79% BD-rate saving when compared with H.266 (VTM).
{"title":"Long-Term and Short-Term Information Propagation and Fusion for Learned Video Compression","authors":"Shen Wang;Donghui Feng;Guo Lu;Zhengxue Cheng;Li Song;Wenjun Zhang","doi":"10.1109/TBC.2024.3434702","DOIUrl":"10.1109/TBC.2024.3434702","url":null,"abstract":"In recent years, numerous learned video compression (LVC) methods have emerged, demonstrating rapid developments and satisfactory performance. However, in most previous methods, only the previous one frame is used as reference. Although some works introduce the usage of the previous multiple frames, the exploitation of temporal information is not comprehensive. Our proposed method not only utilizes the short-term information from multiple neighboring frames but also introduces long-term feature information as the reference, which effectively enhances the quality of the context and improves the compression efficiency. In our scheme, we propose the long-term information exploitation mechanism to capture long-term temporal relevance. The update and propagation of long-term information establish an implicit connection between the latent representation of the current frame and distant reference frames, aiding in the generation of long-term context. Meanwhile, the short-term neighboring frames are also utilized to extract local information and generate short-term context. The fusion of long-term context and short-term context results in a more comprehensive and high-quality context to achieve sufficient temporal information mining. Besides, the multiple frames information also helps to improve the efficiency of motion compression. They are used to generate the predicted motion and remove spatio-temporal redundancies in motion information by second-order motion prediction and fusion. Experimental results demonstrate that our proposed efficient learned video compression scheme can achieve 4.79% BD-rate saving when compared with H.266 (VTM).","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 4","pages":"1254-1265"},"PeriodicalIF":3.2,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The vision of the future 5G - Multicast Broadcast Services (5G-MBS) is to achieve full convergence of broadcast and broadband services by providing these services on the same infrastructure and dynamically switching between them without impacting user experiences. By incorporating Layered Division Multiplexing (LDM) into the new 5G-MBS system and performing proper antenna precoding, the network can transmit a 2-layered signal where the higher-power Core Layer (CL) transmits a Single Frequency Network (SFN) broadcast signal, and the lower-power Enhanced Layer (EL) is used for broadband services. To evaluate the performance of the 2-layered network, a 5G system-level simulator is created and configured according to the 3GPP self-evaluation scenarios to compare against the 3GPP calibration results. The resulting Signal to Interference & Noise Ratio (SINR) Cumulative Distribution Function (CDF) curves fall within the tolerance margin of 1~2 dB from the 3GPP calibration average. Full simulations of the 2-layered network show for an urban scenario with Inter-Site Distance (ISD) of less than 1 km, the network can provide up to three 4K video broadcast services in the CL while supporting a near full broadband network in the EL. For further ISDs up to 3 km, the network can sustain video broadcast service at 1080p while supporting a partial broadband network. For a rural scenario, at the reference ISD of 1732 m, the CL can support three 4k video broadcast services while the EL performance matches a standalone broadband network. Finally, for further ISD of up to 5 km, the CL can support one 1080p and one 720p video broadcast service, and for ISD up to 10 km, the network can provide one broadcast service at 720p in the CL, all while providing a full broadband network in the EL.
{"title":"Layered Division Multiplexing Enabled Broadcast Broadband Convergence in 5G: Theory, Simulations, and Scenarios","authors":"Yu Xue;Wei Li;Yuxiao Zhai;Liang Zhang;Zhihong Hong;Elvino Sousa;Yiyan Wu","doi":"10.1109/TBC.2024.3437204","DOIUrl":"10.1109/TBC.2024.3437204","url":null,"abstract":"The vision of the future 5G - Multicast Broadcast Services (5G-MBS) is to achieve full convergence of broadcast and broadband services by providing these services on the same infrastructure and dynamically switching between them without impacting user experiences. By incorporating Layered Division Multiplexing (LDM) into the new 5G-MBS system and performing proper antenna precoding, the network can transmit a 2-layered signal where the higher-power Core Layer (CL) transmits a Single Frequency Network (SFN) broadcast signal, and the lower-power Enhanced Layer (EL) is used for broadband services. To evaluate the performance of the 2-layered network, a 5G system-level simulator is created and configured according to the 3GPP self-evaluation scenarios to compare against the 3GPP calibration results. The resulting Signal to Interference & Noise Ratio (SINR) Cumulative Distribution Function (CDF) curves fall within the tolerance margin of 1~2 dB from the 3GPP calibration average. Full simulations of the 2-layered network show for an urban scenario with Inter-Site Distance (ISD) of less than 1 km, the network can provide up to three 4K video broadcast services in the CL while supporting a near full broadband network in the EL. For further ISDs up to 3 km, the network can sustain video broadcast service at 1080p while supporting a partial broadband network. For a rural scenario, at the reference ISD of 1732 m, the CL can support three 4k video broadcast services while the EL performance matches a standalone broadband network. Finally, for further ISD of up to 5 km, the CL can support one 1080p and one 720p video broadcast service, and for ISD up to 10 km, the network can provide one broadcast service at 720p in the CL, all while providing a full broadband network in the EL.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 3","pages":"1018-1031"},"PeriodicalIF":3.2,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142226508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-28DOI: 10.1109/tbc.2024.3443544
Hao Yang, Tao Lin, Yuan Zhang, Yin Xu, Zhe Chen, Jinyao Yan
{"title":"Enhancing QoE for Multi-Device Video Delivery: A Novel Dataset and Model Perspective","authors":"Hao Yang, Tao Lin, Yuan Zhang, Yin Xu, Zhe Chen, Jinyao Yan","doi":"10.1109/tbc.2024.3443544","DOIUrl":"https://doi.org/10.1109/tbc.2024.3443544","url":null,"abstract":"","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"13 1","pages":""},"PeriodicalIF":4.5,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-23DOI: 10.1109/TBC.2024.3394293
Qian Huang;Xueguang Yuan;Xiaoyin Yi;Qingming Xie;Qin Jiang;Bingxin Wang
The advent of 5G technology and new energy radio communication systems heralds a significant shift in the landscape of automated driving. This paper focuses on the integration of 5G and broadcast services in the realm of new energy automatic assisted driving, emphasizing the importance of reliable, energy-efficient communication in the large-scale IoV. The enhanced capabilities of 5G enable improved vehicle battery endurance while safeguarding user data privacy and road traffic safety. We introduce RIS relay reflection as a novel approach to optimize non-line-of-sight links, presenting a RIS-assisted communication model tailored for 5G-enhanced large-scale IoV. The paper evaluates the trustworthiness of RIS relays using user behavior data, proposing a reliable and energy-efficient communication scheme that incorporates RIS security relay assistance. This scheme ensures the selection of trustworthy relays, synergizing the beam direction of transmitters and RISs for optimal 5G broadcast service delivery and OTA updates. Our approach promises to revolutionize communication in large-scale IoV systems, paving the way for a more connected and efficient future in automated vehicle networks.
{"title":"Enhancing 5G Broadcast Services in Large-Scale IoV Networks Using Reliable RIS Relaying","authors":"Qian Huang;Xueguang Yuan;Xiaoyin Yi;Qingming Xie;Qin Jiang;Bingxin Wang","doi":"10.1109/TBC.2024.3394293","DOIUrl":"10.1109/TBC.2024.3394293","url":null,"abstract":"The advent of 5G technology and new energy radio communication systems heralds a significant shift in the landscape of automated driving. This paper focuses on the integration of 5G and broadcast services in the realm of new energy automatic assisted driving, emphasizing the importance of reliable, energy-efficient communication in the large-scale IoV. The enhanced capabilities of 5G enable improved vehicle battery endurance while safeguarding user data privacy and road traffic safety. We introduce RIS relay reflection as a novel approach to optimize non-line-of-sight links, presenting a RIS-assisted communication model tailored for 5G-enhanced large-scale IoV. The paper evaluates the trustworthiness of RIS relays using user behavior data, proposing a reliable and energy-efficient communication scheme that incorporates RIS security relay assistance. This scheme ensures the selection of trustworthy relays, synergizing the beam direction of transmitters and RISs for optimal 5G broadcast service delivery and OTA updates. Our approach promises to revolutionize communication in large-scale IoV systems, paving the way for a more connected and efficient future in automated vehicle networks.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 3","pages":"1104-1112"},"PeriodicalIF":3.2,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142207654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-20DOI: 10.1109/TBC.2024.3443469
Kuang-Hsun Lin;Ting-Wei Chen;Hung-Yu Wei
3GPP has established the Multicast Broadcast Services (MBS) standard to accommodate the escalating bandwidth demands of emerging applications like mixed reality and online gaming. MBS offers an efficient means of simultaneously delivering content to different users through the same wireless resources. However, the efficacy of grouping is intricately linked to user mobility and the channel quality of the weakest link. Notably, it is identified that handovers can cause significant interruptions in MBS transmissions. To address this, our paper introduces a novel dynamic grouping scheme capable of adapting to user mobility. Our results demonstrate superior performance compared to state-of-the-art methods without introducing much signaling overhead associated with MBS group management.
{"title":"Mobility-Enabled Dynamic Grouping for Multicast Broadcast Service","authors":"Kuang-Hsun Lin;Ting-Wei Chen;Hung-Yu Wei","doi":"10.1109/TBC.2024.3443469","DOIUrl":"https://doi.org/10.1109/TBC.2024.3443469","url":null,"abstract":"3GPP has established the Multicast Broadcast Services (MBS) standard to accommodate the escalating bandwidth demands of emerging applications like mixed reality and online gaming. MBS offers an efficient means of simultaneously delivering content to different users through the same wireless resources. However, the efficacy of grouping is intricately linked to user mobility and the channel quality of the weakest link. Notably, it is identified that handovers can cause significant interruptions in MBS transmissions. To address this, our paper introduces a novel dynamic grouping scheme capable of adapting to user mobility. Our results demonstrate superior performance compared to state-of-the-art methods without introducing much signaling overhead associated with MBS group management.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"70 4","pages":"1167-1180"},"PeriodicalIF":3.2,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：