{"title":"When guided diffusion model meets zero-shot image super-resolution","authors":"","doi":"10.1016/j.engappai.2024.109336","DOIUrl":null,"url":null,"abstract":"<div><p>Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided <strong>Diff</strong>usion model for <strong>Zero</strong>-shot image SR (<strong>ZeroDiff</strong>) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.</p></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Applications of Artificial Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0952197624014945","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided Diffusion model for Zero-shot image SR (ZeroDiff) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.
期刊介绍:
Artificial Intelligence (AI) is pivotal in driving the fourth industrial revolution, witnessing remarkable advancements across various machine learning methodologies. AI techniques have become indispensable tools for practicing engineers, enabling them to tackle previously insurmountable challenges. Engineering Applications of Artificial Intelligence serves as a global platform for the swift dissemination of research elucidating the practical application of AI methods across all engineering disciplines. Submitted papers are expected to present novel aspects of AI utilized in real-world engineering applications, validated using publicly available datasets to ensure the replicability of research outcomes. Join us in exploring the transformative potential of AI in engineering.