{"title":"Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspective","authors":"Lingchen Sun;Jie Liang;Shuaizheng Liu;Hongwei Yong;Lei Zhang","doi":"10.1109/TIP.2024.3434426","DOIUrl":null,"url":null,"abstract":"High perceptual quality and low distortion degree are two important goals in image restoration tasks such as super-resolution (SR). Most of the existing SR methods aim to achieve these goals by minimizing the corresponding yet conflicting losses, such as the \n<inline-formula> <tex-math>$\\ell _{1}$ </tex-math></inline-formula>\n loss and the adversarial loss. Unfortunately, the commonly used gradient-based optimizers, such as Adam, are hard to balance these objectives due to the opposite gradient decent directions of the contradictory losses. In this paper, we formulate the perception-distortion trade-off in SR as a multi-objective optimization problem and develop a new optimizer by integrating the gradient-free evolutionary algorithm (EA) with gradient-based Adam, where EA and Adam focus on the divergence and convergence of the optimization directions respectively. As a result, a population of optimal models with different perception-distortion preferences is obtained. We then design a fusion network to merge these models into a single stronger one for an effective perception-distortion trade-off. Experiments demonstrate that with the same backbone network, the perception-distortion balanced SR model trained by our method can achieve better perceptual quality than its competitors while attaining better reconstruction fidelity. Codes and models can be found at \n<uri>https://github.com/csslc/EA-Adam</uri>\n.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10620375/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
High perceptual quality and low distortion degree are two important goals in image restoration tasks such as super-resolution (SR). Most of the existing SR methods aim to achieve these goals by minimizing the corresponding yet conflicting losses, such as the
$\ell _{1}$
loss and the adversarial loss. Unfortunately, the commonly used gradient-based optimizers, such as Adam, are hard to balance these objectives due to the opposite gradient decent directions of the contradictory losses. In this paper, we formulate the perception-distortion trade-off in SR as a multi-objective optimization problem and develop a new optimizer by integrating the gradient-free evolutionary algorithm (EA) with gradient-based Adam, where EA and Adam focus on the divergence and convergence of the optimization directions respectively. As a result, a population of optimal models with different perception-distortion preferences is obtained. We then design a fusion network to merge these models into a single stronger one for an effective perception-distortion trade-off. Experiments demonstrate that with the same backbone network, the perception-distortion balanced SR model trained by our method can achieve better perceptual quality than its competitors while attaining better reconstruction fidelity. Codes and models can be found at
https://github.com/csslc/EA-Adam
.
高感知质量和低失真度是超分辨率(SR)等图像复原任务的两个重要目标。现有的大多数 SR 方法都是通过最小化相应但相互冲突的损失(如 ℓ1 损失和对抗损失)来实现这些目标的。遗憾的是,常用的基于梯度的优化器(如 Adam)很难兼顾这些目标,因为相互矛盾的损失具有相反的梯度分布方向。本文将 SR 中的感知-失真权衡问题表述为一个多目标优化问题,并通过将无梯度进化算法(EA)与基于梯度的 Adam 相结合,开发了一种新的优化器,其中 EA 和 Adam 分别关注优化方向的发散性和收敛性。因此,我们得到了具有不同感知失真偏好的最优模型群。然后,我们设计了一个融合网络,将这些模型合并成一个更强的模型,以实现有效的感知-失真权衡。实验证明,在相同的骨干网络下,通过我们的方法训练出的感知-失真平衡的 SR 模型可以获得比竞争对手更好的感知质量,同时达到更好的重建保真度。代码和模型见 https://github.com/csslc/EA-Adam。