{"title":"ZMAR-SNFlow:Restoration for low-light images with massive zero-element pixels","authors":"","doi":"10.1016/j.compeleceng.2024.109750","DOIUrl":null,"url":null,"abstract":"<div><div>Under real-world extremely low-light conditions, many low-light RGB (Red, Green, Blue) images contains massive zero-element pixels (a zero-element pixel is defined as that a color pixel with three RGB values contain no less than one zero). Low-light images with massive zero-element pixels suffer both light weakness and information loss. Existing low-light image enhancement methods aim to amplify the low-light, whereas seldomly consider to restore the information loss caused by massive zero-element pixels. To tackle above issue, firstly, we construct a zero-element mask set that contains many zero-element masks from real-world extremely low-light night traffic monitoring (NTM) images. Each zero-element mask is a binary image, where 1 and 0 are corresponding to zero-element pixels and other pixels. Secondly, we propose a novel flow-based generative method ZMAR-SNFlow to restore low-light images with massive zero-element pixels. ZMAR-SNFlow consists of a zero-element mask attention based Restormer (ZMAR) encoder and a strengthened normalizing flow (SNFlow). Specifically, we proposed a zero-element mask attention (ZMA) module, which is combined with the Restormer module to form the ZMAR module, and ZMAR is used to develop the ZMAR encoder. Then, we propose to insert the unconditional affine coupling layer into the flow step of existing normalizing flow to form SNFlow. ZMAR-SNFlow learns to map the output of SNFlow into a standard normal distribution, and the inverse network of SNFlow takes the latent features of the low-light image as its input to infer the enhanced image. Finally, experimental results on benchmark datasets show that the proposed ZMAR-SNFlow can achieve state-of-the-art (SOTA) performance for low-light images with massive zero-element pixels. The source code and pre-trained models are available at <span><span>https://github.com/NJUPT-IPR-ZhangBo/ZMAR-SNFlow</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50630,"journal":{"name":"Computers & Electrical Engineering","volume":null,"pages":null},"PeriodicalIF":4.0000,"publicationDate":"2024-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers & Electrical Engineering","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0045790624006773","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0
Abstract
Under real-world extremely low-light conditions, many low-light RGB (Red, Green, Blue) images contains massive zero-element pixels (a zero-element pixel is defined as that a color pixel with three RGB values contain no less than one zero). Low-light images with massive zero-element pixels suffer both light weakness and information loss. Existing low-light image enhancement methods aim to amplify the low-light, whereas seldomly consider to restore the information loss caused by massive zero-element pixels. To tackle above issue, firstly, we construct a zero-element mask set that contains many zero-element masks from real-world extremely low-light night traffic monitoring (NTM) images. Each zero-element mask is a binary image, where 1 and 0 are corresponding to zero-element pixels and other pixels. Secondly, we propose a novel flow-based generative method ZMAR-SNFlow to restore low-light images with massive zero-element pixels. ZMAR-SNFlow consists of a zero-element mask attention based Restormer (ZMAR) encoder and a strengthened normalizing flow (SNFlow). Specifically, we proposed a zero-element mask attention (ZMA) module, which is combined with the Restormer module to form the ZMAR module, and ZMAR is used to develop the ZMAR encoder. Then, we propose to insert the unconditional affine coupling layer into the flow step of existing normalizing flow to form SNFlow. ZMAR-SNFlow learns to map the output of SNFlow into a standard normal distribution, and the inverse network of SNFlow takes the latent features of the low-light image as its input to infer the enhanced image. Finally, experimental results on benchmark datasets show that the proposed ZMAR-SNFlow can achieve state-of-the-art (SOTA) performance for low-light images with massive zero-element pixels. The source code and pre-trained models are available at https://github.com/NJUPT-IPR-ZhangBo/ZMAR-SNFlow.
期刊介绍:
The impact of computers has nowhere been more revolutionary than in electrical engineering. The design, analysis, and operation of electrical and electronic systems are now dominated by computers, a transformation that has been motivated by the natural ease of interface between computers and electrical systems, and the promise of spectacular improvements in speed and efficiency.
Published since 1973, Computers & Electrical Engineering provides rapid publication of topical research into the integration of computer technology and computational techniques with electrical and electronic systems. The journal publishes papers featuring novel implementations of computers and computational techniques in areas like signal and image processing, high-performance computing, parallel processing, and communications. Special attention will be paid to papers describing innovative architectures, algorithms, and software tools.