CTI-GAN: Cross-Text-Image Generative Adversarial Network for Bidirectional Cross-modal Generation

Abstract

Cross-modal tasks between text and images are increasingly a research hotspot. This paper proposed a cross-text-image generative adversarial network(CTI-GAN). This model can complete the cross-modal bidirectional generation task between image and text. The method effectively connects text and image modeling to realize bidirectional generation between image and text. The extraction effect of text features is improved by hierarchical LSTM encoding. Through feature pyramid fusion, the features of each layer are fully utilized to improve the image feature representation. In this paper, experiments are conducted to verify the effectiveness of the above improvements for image text generation. The improved algorithm can efficiently complete the task of cross-modal image text generation and improve the accuracy of the generated samples. In the text description generation image task, the inception score of CTI-GAN is improved by about 2% compared with StackGAN++, HDGAN, GAN-INT-CLS, and other models under the same conditions of the same dataset.