How Deep Learning Works for Information Retrieval

Abstract

Information retrieval (IR) is the science of search, the search of user query relevant pieces of information from a collection of unstructured resources. Information in this context includes text, imagery, audio, video, xml, program, and metadata. The journey of an IR process begins with a user query sent to the IR system which encodes the query, compares the query with the available resources, and returns the most relevant pieces of information. Thus, the system is equipped with the ability to store, retrieve and maintain information. In the early era of IR, the whole process was completed using handcrafted features and ad-hoc relevance measures. Later, principled frameworks for relevance measure were developed with statistical learning as a basis. Recently, deep learning has proven essential to the introduction of more opportunities to IR. This is because data-driven features combined with data-driven relevance measures can effectively eliminate the human bias in either feature or relevance measure design. Deep learning has shown its significant potential to transform IR evidenced by abundant empirical results. However, we continue to strive to gain a comprehensive understanding of deep learning. This is done by answering questions such as why deep structures are superior to shallow structures, how skip connections affect a model's performance, uncovering the potential relationship between some of the hyper-parameters and a model's performance, and exploring ways to reduce the chance for deep models to be fooled by adversaries. Answering such questions can help design more effective deep models and devise more efficient schemes for model training.