Know where to go: Make LLM a relevant, responsible, and trustworthy searchers

Abstract

The advent of Large Language Models (LLMs) has shown the potential to improve relevance and provide direct answers in web searches. However, challenges arise in validating the reliability of generated results and the credibility of contributing sources due to the limitations of traditional information retrieval algorithms and the LLM hallucination problem. We aim to transform LLM into a relevant, responsible, and trustworthy searcher in response to these challenges. Rather than following the traditional generative retrieval approach, simply allowing the LLM to summarize the search results, we propose a novel generative retrieval framework leveraging the knowledge of LLMs to foster a direct link between queries and web sources. This framework reforms the retrieval process of the traditional generative retrieval framework by integrating an LLM retriever, and it redesigns the validator while adding an optimizer to ensure the reliability of the retrieved web sources and evidence sentences. Extensive experiments show that our method outperforms several SOTA methods in relevance, responsibility, and trustfulness. It improves search result validity and precision by 2.54 % and 1.05 % over larger-parameter-scale LLM-based systems. Furthermore, it demonstrates significant advantages over traditional frameworks in question-answering and downstream tasks.