Pub Date : 2025-01-23DOI: 10.1109/JETCAS.2025.3533041
Alexandre Mercat;Joose Sainio;Steven Le Moan;Christian Herglotz
High-dynamic range (HDR) video content has gained popularity due to its enhanced color depth and luminance range, but it also presents new challenges in terms of compression efficiency and energy consumption. In this paper, we present an in-depth study of the compression performance and energy efficiency of HDR video encoding using High-Efficiency Video Coding (HEVC). In addition to using a native 10-bit HDR encoding configuration as a reference, we explore whether applying tone mapping to an 8-bit representation before encoding can result in additional bitrate and energy savings without compromising visual quality. The main contributions of this work are as follows: 1) a detailed evaluation of four HDR video encoding configurations, three of which leverage tone mapping techniques, 2) a comprehensive experimental setup involving over 15,000 individual encodings across three open-source HEVC encoders (Kvazaar, x265, and SVT-HEVC) and multiple presets, 3) the use of two advanced perception-based metrics for BD-rate calculations, one of which is specifically tailored to capture colour distortions and 4) an open-source dataset consisting of all experimental results for further research. Among the three tone-mapping configurations tested, our findings show that a simple bit-shifting approach can achieves significant reductions in both bitrate and energy consumption compared to the native 10-bit HDR encoding configuration. This research aims to lay an initial foundation for understanding the balance between coding efficiency and energy consumption in HDR video encoding, offering valuable insights to guide future advancements in the field.
高动态范围（HDR）视频内容因其增强的色彩深度和亮度范围而受到欢迎，但它也在压缩效率和能耗方面提出了新的挑战。本文采用高效视频编码（High-Efficiency video Coding， HEVC）技术对HDR视频编码的压缩性能和能效进行了深入研究。除了使用原生10位HDR编码配置作为参考外，我们还探讨了在编码之前将色调映射应用于8位表示是否可以在不影响视觉质量的情况下节省额外的比特率和能源。本工作的主要贡献如下：1)对四种HDR视频编码配置进行详细评估，其中三种利用色调映射技术；2)在三个开源HEVC编码器（Kvazaar， x265和SVT-HEVC）和多个预设中涉及超过15,000个单独编码的综合实验设置；3)使用两个基于感知的高级指标进行bd率计算；其中一个是专门为捕获颜色失真而定制的，4)一个由所有实验结果组成的开源数据集，用于进一步研究。在测试的三种色调映射配置中，我们的研究结果表明，与原生10位HDR编码配置相比，简单的位移位方法可以显著降低比特率和能耗。本研究旨在为理解HDR视频编码中编码效率和能耗之间的平衡奠定初步基础，为指导该领域的未来发展提供有价值的见解。
{"title":"Do We Need 10 bits? Assessing HEVC Encoders for Energy-Efficient HDR Video Streaming","authors":"Alexandre Mercat;Joose Sainio;Steven Le Moan;Christian Herglotz","doi":"10.1109/JETCAS.2025.3533041","DOIUrl":"https://doi.org/10.1109/JETCAS.2025.3533041","url":null,"abstract":"High-dynamic range (HDR) video content has gained popularity due to its enhanced color depth and luminance range, but it also presents new challenges in terms of compression efficiency and energy consumption. In this paper, we present an in-depth study of the compression performance and energy efficiency of HDR video encoding using High-Efficiency Video Coding (HEVC). In addition to using a native 10-bit HDR encoding configuration as a reference, we explore whether applying tone mapping to an 8-bit representation before encoding can result in additional bitrate and energy savings without compromising visual quality. The main contributions of this work are as follows: 1) a detailed evaluation of four HDR video encoding configurations, three of which leverage tone mapping techniques, 2) a comprehensive experimental setup involving over 15,000 individual encodings across three open-source HEVC encoders (Kvazaar, x265, and SVT-HEVC) and multiple presets, 3) the use of two advanced perception-based metrics for BD-rate calculations, one of which is specifically tailored to capture colour distortions and 4) an open-source dataset consisting of all experimental results for further research. Among the three tone-mapping configurations tested, our findings show that a simple bit-shifting approach can achieves significant reductions in both bitrate and energy consumption compared to the native 10-bit HDR encoding configuration. This research aims to lay an initial foundation for understanding the balance between coding efficiency and energy consumption in HDR video encoding, offering valuable insights to guide future advancements in the field.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"31-43"},"PeriodicalIF":3.7,"publicationDate":"2025-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10851260","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143602036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The significant growth in global video data traffic can be mitigated by saliency-based video coding schemes that seek to increase coding efficiency without any loss of objective visual quality by compressing salient video regions less heavily than non-salient regions. However, conducting salient object detection (SOD) on every video frame before encoding tends to lead to substantial complexity and energy consumption overhead, especially if state-of-the-art deep learning techniques are used in saliency detection. This work introduces a saliency-guided video encoding framework that reduces the energy consumption over frame-by-frame SOD by increasing the detection interval and applying the proposed region-of-interest (ROI) tracking between successive detections. The computational complexity of our ROI tracking technique is kept low by predicting object movements from motion vectors, which are inherently calculated during encoding. Our experimental results demonstrate that the proposed ROI tracking solution saves energy by 86-95% and attains 84-94% accuracy over frame-by-frame SOD. Correspondingly, integrating our proposal into the complete saliency-guided video coding scheme reduces energy consumption on CPU by 79-82% at a cost of weighted PSNR of less than 5%. These findings indicate that our solution has significant potential for low-cost and low-power streaming media applications.
{"title":"Energy-Efficient Saliency-Guided Video Coding Framework for Real-Time Applications","authors":"Tero Partanen;Minh Hoang;Alexandre Mercat;Joose Sainio;Jarno Vanne","doi":"10.1109/JETCAS.2024.3525339","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3525339","url":null,"abstract":"The significant growth in global video data traffic can be mitigated by saliency-based video coding schemes that seek to increase coding efficiency without any loss of objective visual quality by compressing salient video regions less heavily than non-salient regions. However, conducting salient object detection (SOD) on every video frame before encoding tends to lead to substantial complexity and energy consumption overhead, especially if state-of-the-art deep learning techniques are used in saliency detection. This work introduces a saliency-guided video encoding framework that reduces the energy consumption over frame-by-frame SOD by increasing the detection interval and applying the proposed region-of-interest (ROI) tracking between successive detections. The computational complexity of our ROI tracking technique is kept low by predicting object movements from motion vectors, which are inherently calculated during encoding. Our experimental results demonstrate that the proposed ROI tracking solution saves energy by 86-95% and attains 84-94% accuracy over frame-by-frame SOD. Correspondingly, integrating our proposal into the complete saliency-guided video coding scheme reduces energy consumption on CPU by 79-82% at a cost of weighted PSNR of less than 5%. These findings indicate that our solution has significant potential for low-cost and low-power streaming media applications.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"44-57"},"PeriodicalIF":3.7,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10820524","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143601935","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-30DOI: 10.1109/JETCAS.2024.3524260
Peilin Chen;Xiaohan Fang;Meng Wang;Shiqi Wang;Siwei Ma
The Human Visual System (HVS), with its intricate sophistication, is capable of achieving ultra-compact information compression for visual signals. This remarkable ability is coupled with high generalization capability and energy efficiency. By contrast, the state-of-the-art Versatile Video Coding (VVC) standard achieves a compression ratio of around 1,000 times for raw visual data. This notable disparity motivates the research community to draw inspiration to effectively handle the immense volume of visual data in a green way. Therefore, this paper provides a survey of how visual data can be efficiently represented for green multimedia, in particular when the ultimate task is knowledge extraction instead of visual signal reconstruction. We introduce recent research efforts that promote green, sustainable, and efficient multimedia in this field. Moreover, we discuss how the deep understanding of the HVS can benefit the research community, and envision the development of future green multimedia technologies.
{"title":"Compact Visual Data Representation for Green Multimedia–A Human Visual System Perspective","authors":"Peilin Chen;Xiaohan Fang;Meng Wang;Shiqi Wang;Siwei Ma","doi":"10.1109/JETCAS.2024.3524260","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3524260","url":null,"abstract":"The Human Visual System (HVS), with its intricate sophistication, is capable of achieving ultra-compact information compression for visual signals. This remarkable ability is coupled with high generalization capability and energy efficiency. By contrast, the state-of-the-art Versatile Video Coding (VVC) standard achieves a compression ratio of around 1,000 times for raw visual data. This notable disparity motivates the research community to draw inspiration to effectively handle the immense volume of visual data in a green way. Therefore, this paper provides a survey of how visual data can be efficiently represented for green multimedia, in particular when the ultimate task is knowledge extraction instead of visual signal reconstruction. We introduce recent research efforts that promote green, sustainable, and efficient multimedia in this field. Moreover, we discuss how the deep understanding of the HVS can benefit the research community, and envision the development of future green multimedia technologies.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"16-30"},"PeriodicalIF":3.7,"publicationDate":"2024-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143602038","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-25DOI: 10.1109/JETCAS.2024.3523246
Daiane Freitas;Patrick Rosa;Leonardo Müller;Daniel Palomino;Cláudio M. Diniz;Mateus Grellert;Guilherme Corrêa
In modern video encoders, sub-pixel motion models are used to represent smoother transitions between neighboring frames, which is specially useful in regions with intense movement. The AV1 video codec introduces adaptive filtering for sub-pixel interpolation in the inter-frame prediction stage, enhancing flexibility in Motion Estimation (ME) and Motion Compensation (MC), using three filter types: Regular, Sharp, and Smooth. However, the increased variety of filters leads to higher complexity and energy consumption, particularly during the resource-intensive generation of sub-pixel samples. To address this challenge, this paper presents a hardware accelerator optimized for AV1 interpolation, incorporating energy-saving features for unused filters. The accelerator includes one precise version that can be used for both MC and ME and two approximate versions for ME, designed to maximize hardware efficiency and minimize implementation costs. The proposed design can process videos at resolutions up to 4320p at 50 frames per second for MC and 2,656.14 million samples per second for ME, with a power dissipation ranging between 21.25 mW and 40.06 mW, and an average coding efficiency loss of 0.67% and 1.11%, depending on the filter type and version.
{"title":"Low-Power Multiversion Interpolation Filter Accelerator With Hardware Reuse for AV1 Codec","authors":"Daiane Freitas;Patrick Rosa;Leonardo Müller;Daniel Palomino;Cláudio M. Diniz;Mateus Grellert;Guilherme Corrêa","doi":"10.1109/JETCAS.2024.3523246","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3523246","url":null,"abstract":"In modern video encoders, sub-pixel motion models are used to represent smoother transitions between neighboring frames, which is specially useful in regions with intense movement. The AV1 video codec introduces adaptive filtering for sub-pixel interpolation in the inter-frame prediction stage, enhancing flexibility in Motion Estimation (ME) and Motion Compensation (MC), using three filter types: Regular, Sharp, and Smooth. However, the increased variety of filters leads to higher complexity and energy consumption, particularly during the resource-intensive generation of sub-pixel samples. To address this challenge, this paper presents a hardware accelerator optimized for AV1 interpolation, incorporating energy-saving features for unused filters. The accelerator includes one precise version that can be used for both MC and ME and two approximate versions for ME, designed to maximize hardware efficiency and minimize implementation costs. The proposed design can process videos at resolutions up to 4320p at 50 frames per second for MC and 2,656.14 million samples per second for ME, with a power dissipation ranging between 21.25 mW and 40.06 mW, and an average coding efficiency loss of 0.67% and 1.11%, depending on the filter type and version.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"133-142"},"PeriodicalIF":3.7,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143602016","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-13DOI: 10.1109/JETCAS.2024.3502893
{"title":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems Information for Authors","authors":"","doi":"10.1109/JETCAS.2024.3502893","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3502893","url":null,"abstract":"","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 4","pages":"835-835"},"PeriodicalIF":3.7,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10799918","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-13DOI: 10.1109/JETCAS.2024.3502897
{"title":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems Publication Information","authors":"","doi":"10.1109/JETCAS.2024.3502897","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3502897","url":null,"abstract":"","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 4","pages":"C2-C2"},"PeriodicalIF":3.7,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10799919","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Presents corrections to the paper, (Erratum to “A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks”).
提出对论文的更正（"A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks "的勘误）。
{"title":"Erratum to “A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks”","authors":"Lichuan Luo;Wang Kang;Junzhan Liu;He Zhang;Youguang Zhang;Dijun Liu;Peng Ouyang","doi":"10.1109/JETCAS.2024.3464190","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3464190","url":null,"abstract":"Presents corrections to the paper, (Erratum to “A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks”).","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 4","pages":"834-834"},"PeriodicalIF":3.7,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10799921","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-13DOI: 10.1109/JETCAS.2024.3502895
{"title":"IEEE Circuits and Systems Society Information","authors":"","doi":"10.1109/JETCAS.2024.3502895","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3502895","url":null,"abstract":"","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 4","pages":"C3-C3"},"PeriodicalIF":3.7,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10799541","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Guest Editorial: Toward Trustworthy AI: Advances in Circuits, Systems, and Applications","authors":"Shih-Hsu Huang;Pin-Yu Chen;Stjepan Picek;Chip-Hong Chang","doi":"10.1109/JETCAS.2024.3497232","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3497232","url":null,"abstract":"","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"14 4","pages":"577-581"},"PeriodicalIF":3.7,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10799920","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-11DOI: 10.1109/JETCAS.2024.3515055
Azin Izadi;Vahid Jamshidi
Numerical computations in various applications can often tolerate a small degree of error. In fields such as data mining, encoding algorithms, image processing, machine learning, and signal processing where error resilience is crucial approximate computing can effectively replace precise computing to minimize circuit delay and power consumption. In these contexts, a certain level of error is permissible. Multiplication, a fundamental arithmetic operation in computer systems, often leads to increased circuit delay, power usage, and area occupation when performed accurately by multipliers, which are key components in these applications. Thus, developing an optimal multiplier represents a significant advantage for inexact computing systems. In this paper, we introduce a novel approximate multiplier based on the Mitchell algorithm. The proposed design has been implemented using the Cadence software environment with the TSMC 45nm standard-cell library and a supply voltage of 1.1V. Simulation results demonstrate an average reduction of 31.7% in area, 46.8% in power consumption, and 36.1% in circuit delay compared to previous works. The mean relative error distance (MRED) for the proposed method is recorded at 2.6%.
{"title":"LSHIM: Low-Power and Small-Area Inexact Multiplier for High-Speed Error-Resilient Applications","authors":"Azin Izadi;Vahid Jamshidi","doi":"10.1109/JETCAS.2024.3515055","DOIUrl":"https://doi.org/10.1109/JETCAS.2024.3515055","url":null,"abstract":"Numerical computations in various applications can often tolerate a small degree of error. In fields such as data mining, encoding algorithms, image processing, machine learning, and signal processing where error resilience is crucial approximate computing can effectively replace precise computing to minimize circuit delay and power consumption. In these contexts, a certain level of error is permissible. Multiplication, a fundamental arithmetic operation in computer systems, often leads to increased circuit delay, power usage, and area occupation when performed accurately by multipliers, which are key components in these applications. Thus, developing an optimal multiplier represents a significant advantage for inexact computing systems. In this paper, we introduce a novel approximate multiplier based on the Mitchell algorithm. The proposed design has been implemented using the Cadence software environment with the TSMC 45nm standard-cell library and a supply voltage of 1.1V. Simulation results demonstrate an average reduction of 31.7% in area, 46.8% in power consumption, and 36.1% in circuit delay compared to previous works. The mean relative error distance (MRED) for the proposed method is recorded at 2.6%.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"94-104"},"PeriodicalIF":3.7,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143602011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：