Contrastive learning for a single historical painting’s blind super-resolution

IF 3.8 3区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS Visual Informatics Pub Date : 2021-12-01 DOI:10.1016/j.visinf.2021.11.002
Hongzhen Shi, Dan Xu, Kangjian He, Hao Zhang, Yingying Yue
{"title":"Contrastive learning for a single historical painting’s blind super-resolution","authors":"Hongzhen Shi,&nbsp;Dan Xu,&nbsp;Kangjian He,&nbsp;Hao Zhang,&nbsp;Yingying Yue","doi":"10.1016/j.visinf.2021.11.002","DOIUrl":null,"url":null,"abstract":"<div><p>Most of the existing blind super-resolution(SR) methods explicitly estimate the kernel in pixel space, which usually has a large deviation and results in poor SR performance. As a seminal work, DASR learns abstract representations to distinguish various degradations in the feature space, which effectively reduces degradation estimation bias. Therefore, we also employ the feature space to extract degradation representations for an ancient painting. However, most of the blind SR mehods, including DASR, are committed to removing degradations introduced by kernels, downsampling and additive noise. Among them, downsampling degradation is often accompanied by unpleasant artifacts. To address this issue, the paper designs a high-resolution(HR) representation encoder <span><math><msub><mrow><mi>E</mi></mrow><mrow><mi>H</mi><mi>R</mi></mrow></msub></math></span> based on contrastive learning to distinguish artifacts introduced by downsampling. Moreover, to optimize the ill-posed nature of blind SR, we propose a contrastive regularization<span><math><mrow><mo>(</mo><mi>C</mi><mi>R</mi><mo>)</mo></mrow></math></span> to minimize the contrastive loss based on VGG-19. With the help of <span><math><mrow><mi>C</mi><mi>R</mi></mrow></math></span>, the SR images are pulled closer to the HR images and pushed far away from bicubic LR observations. Benefiting from these improvements, our method consistently achieves higher quantitative performance and better visual quality with more natural textures than state-of-the-art approaches on a specialized painting dataset.</p></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":null,"pages":null},"PeriodicalIF":3.8000,"publicationDate":"2021-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2468502X21000504/pdfft?md5=251aa45f07cdb0c2ad16c7a5a2ac5156&pid=1-s2.0-S2468502X21000504-main.pdf","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Visual Informatics","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2468502X21000504","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 1

Abstract

Most of the existing blind super-resolution(SR) methods explicitly estimate the kernel in pixel space, which usually has a large deviation and results in poor SR performance. As a seminal work, DASR learns abstract representations to distinguish various degradations in the feature space, which effectively reduces degradation estimation bias. Therefore, we also employ the feature space to extract degradation representations for an ancient painting. However, most of the blind SR mehods, including DASR, are committed to removing degradations introduced by kernels, downsampling and additive noise. Among them, downsampling degradation is often accompanied by unpleasant artifacts. To address this issue, the paper designs a high-resolution(HR) representation encoder EHR based on contrastive learning to distinguish artifacts introduced by downsampling. Moreover, to optimize the ill-posed nature of blind SR, we propose a contrastive regularization(CR) to minimize the contrastive loss based on VGG-19. With the help of CR, the SR images are pulled closer to the HR images and pushed far away from bicubic LR observations. Benefiting from these improvements, our method consistently achieves higher quantitative performance and better visual quality with more natural textures than state-of-the-art approaches on a specialized painting dataset.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
单幅历史画盲目超分辨的对比学习
现有的盲超分辨(SR)方法大多是在像素空间中显式地估计核,通常存在较大的偏差,导致SR性能较差。作为一项开创性的工作,DASR学习抽象表征来区分特征空间中的各种退化,有效地减少了退化估计偏差。因此,我们也采用特征空间来提取古画的退化表征。然而,大多数盲SR方法,包括DASR,都致力于去除核、下采样和加性噪声带来的退化。其中,下采样退化通常伴随着令人不快的伪影。为了解决这一问题,本文设计了一种基于对比学习的高分辨率(HR)表示编码器EHR,以区分下采样引入的伪影。此外,为了优化盲SR的病态性,我们提出了一种基于VGG-19的对比正则化(CR)来最小化对比损失。在CR的帮助下,SR图像更接近HR图像,并远离双立方LR观测。得益于这些改进,我们的方法与专业绘画数据集上最先进的方法相比,始终如一地实现更高的定量性能和更好的视觉质量,具有更自然的纹理。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Visual Informatics
Visual Informatics Computer Science-Computer Graphics and Computer-Aided Design
CiteScore
6.70
自引率
3.30%
发文量
33
审稿时长
79 days
期刊最新文献
RelicCARD: Enhancing cultural relics exploration through semantics-based augmented reality tangible interaction design JobViz: Skill-driven visual exploration of job advertisements Visual evaluation of graph representation learning based on the presentation of community structures DPKnob: A visual analysis approach to risk-aware formulation of differential privacy schemes for data query scenarios Visual exploration of multi-dimensional data via rule-based sample embedding
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1