基于生成对抗网络的文本到图像的研究

Li Xiaolin, Gao Yuwei
{"title":"基于生成对抗网络的文本到图像的研究","authors":"Li Xiaolin, Gao Yuwei","doi":"10.1109/ITCA52113.2020.00077","DOIUrl":null,"url":null,"abstract":"In recent years, Generative Adversarial Network (GAN) has quickly become the most popular deep generative model framework, and it is also the most popular topic in the current deep learning research field. Although the generative adversarial network has achieved remarkable results from text description to image generation, when a complex image containing multiple objects, the position of each object will be blurred and overlapped, and the edges of the generated image will be blurred and local textures will be unclear. Usually given text description can generate the corresponding rough image, but there are still some problems in the image details. In order to solve the above problems, on the basis of Stack GAN, a scene graph-based stacked generative confrontation network model (Scene graph stack GAN, SGS-GAN) is proposed, which converts the text description into The scene graph uses the scene graph as the condition vector and inputs the random noise into the generator model to obtain the result image. The experimental results show that the Inception store of the SGS-GAN model on the Visual Genome and COCO data sets reached 6.64 and 6.52, respectively, which were increased by 0.212 and 0.219 compared to Sg2Im. This proves that the diversity and vividness of the generated samples and the sharpness of the image are obviously improved after the number of times of training and the input of the scene graph.","PeriodicalId":103309,"journal":{"name":"2020 2nd International Conference on Information Technology and Computer Application (ITCA)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Research on Text to Image Based on Generative Adversarial Network\",\"authors\":\"Li Xiaolin, Gao Yuwei\",\"doi\":\"10.1109/ITCA52113.2020.00077\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In recent years, Generative Adversarial Network (GAN) has quickly become the most popular deep generative model framework, and it is also the most popular topic in the current deep learning research field. Although the generative adversarial network has achieved remarkable results from text description to image generation, when a complex image containing multiple objects, the position of each object will be blurred and overlapped, and the edges of the generated image will be blurred and local textures will be unclear. Usually given text description can generate the corresponding rough image, but there are still some problems in the image details. In order to solve the above problems, on the basis of Stack GAN, a scene graph-based stacked generative confrontation network model (Scene graph stack GAN, SGS-GAN) is proposed, which converts the text description into The scene graph uses the scene graph as the condition vector and inputs the random noise into the generator model to obtain the result image. The experimental results show that the Inception store of the SGS-GAN model on the Visual Genome and COCO data sets reached 6.64 and 6.52, respectively, which were increased by 0.212 and 0.219 compared to Sg2Im. This proves that the diversity and vividness of the generated samples and the sharpness of the image are obviously improved after the number of times of training and the input of the scene graph.\",\"PeriodicalId\":103309,\"journal\":{\"name\":\"2020 2nd International Conference on Information Technology and Computer Application (ITCA)\",\"volume\":\"2 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2020-12-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2020 2nd International Conference on Information Technology and Computer Application (ITCA)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ITCA52113.2020.00077\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2020 2nd International Conference on Information Technology and Computer Application (ITCA)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ITCA52113.2020.00077","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1

摘要

近年来,生成对抗网络(Generative Adversarial Network, GAN)迅速成为最流行的深度生成模型框架,也是当前深度学习研究领域最热门的话题。虽然生成式对抗网络从文本描述到图像生成都取得了显著的效果,但是当一个复杂的图像包含多个物体时,每个物体的位置会被模糊和重叠,生成的图像的边缘会模糊,局部纹理会不清晰。通常给出的文字描述可以生成相应的粗糙图像,但在图像细节方面还存在一些问题。为了解决上述问题,在Stack GAN的基础上,提出了一种基于场景图的堆叠生成对抗网络模型(scene graph Stack GAN, SGS-GAN),该模型将文本描述转换为场景图,以场景图为条件向量,将随机噪声输入到生成器模型中,得到结果图像。实验结果表明,该模型在Visual Genome和COCO数据集上的Inception store分别达到6.64和6.52,比Sg2Im分别提高了0.212和0.219。这证明经过多次训练和场景图的输入后,生成的样本的多样性、生动性和图像的清晰度都有了明显的提高。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Research on Text to Image Based on Generative Adversarial Network
In recent years, Generative Adversarial Network (GAN) has quickly become the most popular deep generative model framework, and it is also the most popular topic in the current deep learning research field. Although the generative adversarial network has achieved remarkable results from text description to image generation, when a complex image containing multiple objects, the position of each object will be blurred and overlapped, and the edges of the generated image will be blurred and local textures will be unclear. Usually given text description can generate the corresponding rough image, but there are still some problems in the image details. In order to solve the above problems, on the basis of Stack GAN, a scene graph-based stacked generative confrontation network model (Scene graph stack GAN, SGS-GAN) is proposed, which converts the text description into The scene graph uses the scene graph as the condition vector and inputs the random noise into the generator model to obtain the result image. The experimental results show that the Inception store of the SGS-GAN model on the Visual Genome and COCO data sets reached 6.64 and 6.52, respectively, which were increased by 0.212 and 0.219 compared to Sg2Im. This proves that the diversity and vividness of the generated samples and the sharpness of the image are obviously improved after the number of times of training and the input of the scene graph.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Research on Physically-based Computer Animation Application and research of heuristic search algorithm in crawler field Application of Student achievement Analysis based on Apriori Algorithm A Robust Routing Algorithm with Dynamic Minimum Hop Selection in Wireless Sensor Networks with Unreliable Links Research on Formation Pressure Prediction Based on Neural Network System Identification Theory
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1