Transferring Markov Network for Information Retrieval

Abstract

Along with the development of internet, a lot of new data appears in the web every day. To construct a retrieval model to adapt the new data quickly and to retrieval the new documents accurately is becoming an important research topic. In this paper, we put forward a new retrieval model by incorporating the theory of transfer learning with Markov Network. Firstly, compare term spaces network of old dataset and new (target) dataset, and the distance between data sets is measured using the Kullback-Leibler divergence. Moreover, KL-divergence is used to decide the trade-off parameter in retrieval formula. Then we transfer the useful prior knowledge of old dataset to the new (target) dataset, and finally implement the retrieval process on the target dataset. Experiments on multiple datasets indicate that our new approach outperforms other methods. Furthermore, we perform several T-tests to demonstrate the improvements are statistically significant.