The Design and Research of Cross-media Retrieval

Abstract

Nowadays, the rapid development of Internet technology drives the advance of the big data era, and along with the widespread use of smart devices and social media, multimedia data is growing explosively. With the increasingly complex and diverse needs of information exchange, collection and storage, the types of information have also evolved from traditional text information to diverse data forms such as pictures and video and audio, bringing different degrees of convenience to people's work and life and other scenarios. However, the huge amount of multimedia data also makes information storage and retrieval more cumbersome. How to realize the effective storage and efficient retrieval of data, so as to better utilize the value of multimedia data, is one of the challenges that academia and information industry are tackling nowadays. In this paper, we study the cross-media retrieval technology of text and image by Contrastive Language-Image Pre-training model based on natural language processing method. The cross-media pre-training idea proposed in this paper can be applied not only to text-image processing, but also theoretically to mutual retrieval of modal information of video and audio, etc.