面向内容感知的多属性编辑潜在语义方向融合

IF 2.3 4区 计算机科学 Q2 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE IEEE MultiMedia Pub Date : 2023-07-01 DOI:10.1109/MMUL.2023.3285550
Xiwen Wei, Yihan Tang, Si Wu
{"title":"面向内容感知的多属性编辑潜在语义方向融合","authors":"Xiwen Wei, Yihan Tang, Si Wu","doi":"10.1109/MMUL.2023.3285550","DOIUrl":null,"url":null,"abstract":"For facial attribute editing, significant progress has been made in discovering semantic directions in the latent space of StyleGAN, and the manipulation is performed by mapping an input image to a latent code and then moving along a direction associated with a target attribute. In this case, multi-attribute editing typically needs a sequential transformation process, which may cause ineffective manipulation or the cumulative effect on irrelevant attribute deviation. In this work, we aim to simultaneously manipulate multiple attributes through a single transformation. Toward this end, we propose a StyleGAN-based latent semantic direction fusion model, referred to as StyleLSF. There are two learnable components: a content-aware direction predictor learns to infer the latent directions, which are associated with preset attributes. A fusion network fuses the directions with respect to target attributes and yields a single translation vector. We further ensure irrelevant attribute preservation by imposing an attribute-aware feature consistency regularization approach.","PeriodicalId":13240,"journal":{"name":"IEEE MultiMedia","volume":"30 1","pages":"87-99"},"PeriodicalIF":2.3000,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Content-Aware Latent Semantic Direction Fusion for Multi-Attribute Editing\",\"authors\":\"Xiwen Wei, Yihan Tang, Si Wu\",\"doi\":\"10.1109/MMUL.2023.3285550\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"For facial attribute editing, significant progress has been made in discovering semantic directions in the latent space of StyleGAN, and the manipulation is performed by mapping an input image to a latent code and then moving along a direction associated with a target attribute. In this case, multi-attribute editing typically needs a sequential transformation process, which may cause ineffective manipulation or the cumulative effect on irrelevant attribute deviation. In this work, we aim to simultaneously manipulate multiple attributes through a single transformation. Toward this end, we propose a StyleGAN-based latent semantic direction fusion model, referred to as StyleLSF. There are two learnable components: a content-aware direction predictor learns to infer the latent directions, which are associated with preset attributes. A fusion network fuses the directions with respect to target attributes and yields a single translation vector. We further ensure irrelevant attribute preservation by imposing an attribute-aware feature consistency regularization approach.\",\"PeriodicalId\":13240,\"journal\":{\"name\":\"IEEE MultiMedia\",\"volume\":\"30 1\",\"pages\":\"87-99\"},\"PeriodicalIF\":2.3000,\"publicationDate\":\"2023-07-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE MultiMedia\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1109/MMUL.2023.3285550\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE MultiMedia","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1109/MMUL.2023.3285550","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0

摘要

对于面部属性编辑,在StyleGAN的潜在空间中发现语义方向方面已经取得了重大进展,并且通过将输入图像映射到潜在代码,然后沿着与目标属性相关联的方向移动来执行操作。在这种情况下,多属性编辑通常需要一个顺序转换过程,这可能会导致无效的操作或对无关属性偏差的累积影响。在这项工作中,我们的目标是通过单个转换同时操作多个属性。为此,我们提出了一个基于StyleGAN的潜在语义方向融合模型,称为StyleSF。有两个可学习的组成部分:内容感知方向预测器学习推断与预设属性相关的潜在方向。融合网络融合相对于目标属性的方向,并产生单个平移向量。我们通过采用属性感知特征一致性正则化方法来进一步确保不相关的属性保持。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Content-Aware Latent Semantic Direction Fusion for Multi-Attribute Editing
For facial attribute editing, significant progress has been made in discovering semantic directions in the latent space of StyleGAN, and the manipulation is performed by mapping an input image to a latent code and then moving along a direction associated with a target attribute. In this case, multi-attribute editing typically needs a sequential transformation process, which may cause ineffective manipulation or the cumulative effect on irrelevant attribute deviation. In this work, we aim to simultaneously manipulate multiple attributes through a single transformation. Toward this end, we propose a StyleGAN-based latent semantic direction fusion model, referred to as StyleLSF. There are two learnable components: a content-aware direction predictor learns to infer the latent directions, which are associated with preset attributes. A fusion network fuses the directions with respect to target attributes and yields a single translation vector. We further ensure irrelevant attribute preservation by imposing an attribute-aware feature consistency regularization approach.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE MultiMedia
IEEE MultiMedia 工程技术-计算机:理论方法
CiteScore
6.40
自引率
3.10%
发文量
59
审稿时长
>12 weeks
期刊介绍: The magazine contains technical information covering a broad range of issues in multimedia systems and applications. Articles discuss research as well as advanced practice in hardware/software and are expected to span the range from theory to working systems. Especially encouraged are papers discussing experiences with new or advanced systems and subsystems. To avoid unnecessary overlap with existing publications, acceptable papers must have a significant focus on aspects unique to multimedia systems and applications. These aspects are likely to be related to the special needs of multimedia information compared to other electronic data, for example, the size requirements of digital media and the importance of time in the representation of such media. The following list is not exhaustive, but is representative of the topics that are covered: Hardware and software for media compression, coding & processing; Media representations & standards for storage, editing, interchange, transmission & presentation; Hardware platforms supporting multimedia applications; Operating systems suitable for multimedia applications; Storage devices & technologies for multimedia information; Network technologies, protocols, architectures & delivery techniques intended for multimedia; Synchronization issues; Multimedia databases; Formalisms for multimedia information systems & applications; Programming paradigms & languages for multimedia; Multimedia user interfaces; Media creation integration editing & management; Creation & modification of multimedia applications.
期刊最新文献
Generative Adversarial Networks for Biomedical Imaging High-performance Embedded System Design for QR Code Recognition with Deep Learning Terrain Segmentation Network in Wild Environments with Hybrid Plus Downsampling Robust Color Image Hashing with NMF and Saliency Map for Copy Detection Development of an Image Encryption Algorithm Based on Compressed Sensing and Chaotic Mapping
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1