Foundations and Trends in Information Retrieval最新文献

Gender bias is a pervasive issue that continues to influence various aspects of society, including the outcomes of information retrieval (IR) systems. As these systems become increasingly integral to accessing and navigating the vast amounts of information available today, the need to understand and mitigate gender bias within them is paramount. This monograph provides a comprehensive examination of the origins, manifestations, and consequences of gender bias in IR systems, as well as the current methodologies employed to address these biases.

Theoretical frameworks surrounding gender and its representation in artificial intelligence (AI) systems are explored, particularly focusing on how traditional gender binaries are perpetuated and reinforced through data and algorithmic processes. Metrics and methodologies used to identify and measure gender bias within IR systems are then analyzed, offering a detailed evaluation of existing approaches and their limitations.

Subsequent sections address the sources of gender bias, including biased input queries, retrieval methods, and gold standard datasets. Various data-driven and method-level debiasing strategies are presented, including techniques for debiasing neural embeddings and algorithmic approaches aimed at reducing bias in IR system outputs. The monograph concludes with a discussion of the challenges and limitations faced by current debiasing efforts and provides insights into future research directions that could lead to more equitable and inclusive IR systems.

This monograph serves as a valuable resource for researchers, practitioners, and students in the fields of information retrieval, artificial intelligence, and data science, providing the knowledge and tools needed to address gender bias and contribute to the development of fair and unbiased information systems.

Mathematical information is essential for technical work, but its creation, interpretation, and search are challenging. To help address these challenges, researchers have developed multimodal search engines and mathematical question answering systems. This monograph begins with a simple framework characterizing the information tasks that people and systems perform as we work to answer math-related questions. The framework is used to organize and relate the other core topics of the monograph, including interactions between people and systems, representing math formulas in sources, and evaluation. We close by addressing some key questions and presenting directions for future work. This monograph is intended for students, instructors, and researchers interested in systems that help us find and use mathematical information.

Electronic commerce, or e-commerce, is the buying and selling of goods and services, or the transmitting of funds or data online. E-commerce platforms come in many kinds, with global players such as Amazon, Airbnb, Alibaba, Booking.com, eBay, and JD.com and platforms targeting specific geographic regions such as Bol.com and Flipkart.com. Information retrieval has a natural role to play in e-commerce, especially in connecting people to goods and services. Information discovery in e-commerce concerns different types of search (e.g., exploratory search vs. lookup tasks), recommender systems, and natural language processing in e-commerce portals. The rise in popularity of e-commerce sites has made research on information discovery in e-commerce an increasingly active research area. This is witnessed by an increase in publications and dedicated workshops in this space. Methods for information discovery in e-commerce largely focus on improving the effectiveness of e-commerce search and recommender systems, on enriching and using knowledge graphs to support e-commerce, and on developing innovative question answering and bot-based solutions that help to connect people to goods and services. In this survey, an overview is given of the fundamental infrastructure, algorithms, and technical solutions for information discovery in e-commerce. The topics covered include user behavior and profiling, search, recommendation, and language technology in e-commerce.

Search engines play a crucial role in organizing and deliveringinformation to billions of users worldwide. However,these systems often reflect and amplify existing societalbiases and stereotypes through their search results and rankings.This concern has prompted researchers to investigatemethods for measuring and reducing algorithmic bias, withthe goal of developing more equitable search systems. Thismonograph presents a comprehensive taxonomy of fairnessin search systems and surveys the current research landscape.We systematically examine how bias manifests acrosskey search components, including query interpretation andprocessing, document representation and indexing, resultranking algorithms, and system evaluation metrics. By criticallyanalyzing the existing literature, we identify persistentchallenges and promising research directions in the pursuitof fairer search systems. Our aim is to provide a foundationfor future work in this rapidly evolving field while highlightingopportunities to create more inclusive and equitableinformation retrieval technologies.

Information access systems, such as search engines, recommendersystems, and conversational assistants, have becomeintegral to our daily lives as they help us satisfy our informationneeds. However, evaluating the effectiveness ofthese systems presents a long-standing and complex scientificchallenge. This challenge is rooted in the difficulty ofassessing a system’s overall effectiveness in assisting usersto complete tasks through interactive support, and furtherexacerbated by the substantial variation in user behaviourand preferences. To address this challenge, user simulationemerges as a promising solution.

This monograph focuses on providing a thorough understandingof user simulation techniques designed specificallyfor evaluation purposes. We begin with a background of informationaccess system evaluation and explore the diverseapplications of user simulation. Subsequently, we systematicallyreview the major research progress in user simulation,covering both general frameworks for designing user simulators,utilizing user simulation for evaluation, and specificmodels and algorithms for simulating user interactions withsearch engines, recommender systems, and conversationalassistants. Realizing that user simulation is an interdisciplinaryresearch topic, whenever possible, we attempt toestablish connections with related fields, including machinelearning, dialogue systems, user modeling, and economics.We end the monograph with a broad discussion of importantfuture research directions, many of which extend beyond theevaluation of information access systems and are expectedto have broader impact on how to evaluate interactive intelligentsystems in general.

The task of Question Answering (QA) has attracted significantresearch interest for a long time. Its relevance tolanguage understanding and knowledge retrieval tasks, alongwith the simple setting, makes the task of QA crucial forstrong AI systems. Recent success on simple QA tasks hasshifted the focus to more complex settings. Among these,Multi-Hop QA (MHQA) is one of the most researched tasksover recent years. In broad terms, MHQA is the task of answeringnatural language questions that involve extractingand combining multiple pieces of information and doing multiplesteps of reasoning. An example of a multi-hop questionwould be “The Argentine PGA Championship record holderhas won how many tournaments worldwide?”. Answeringthe question would need two pieces of information: “Who isthe record holder for Argentine PGA Championship tournaments?”and “How many tournaments did [Answer of SubQ1] win?”. The ability to answer multi-hop questions andperform multi step reasoning can significantly improve theutility of NLP systems. Consequently, the field has seen asurge of high quality datasets, models and evaluation strategies.The notion of ‘multiple hops’ is somewhat abstractwhich results in a large variety of tasks that require multihopreasoning. This leads to different datasets and modelsthat differ significantly from each other and make the fieldchallenging to generalize and survey. We aim to provide ageneral and formal definition of the MHQA task, and organizeand summarize existing MHQA frameworks. We alsooutline some best practices for building MHQA datasets.This monograph provides a systematic and thorough introductionas well as the structuring of the existing attemptsto this highly interesting, yet quite challenging task.

Conversational information seeking (CIS) is concerned with a sequence of interactions between one or more users and an information system. Interactions in CIS are primarily based on natural language dialogue, while they may include other types of interactions, such as click, touch, and body gestures. This monograph provides a thorough overview of CIS definitions, applications, interactions, interfaces, design, implementation, and evaluation. This monograph views CIS applications as including conversational search, conversational question answering, and conversational recommendation. Our aim is to provide an overview of past research related to CIS, introduce the current state-of-the-art in CIS, highlight the challenges still being faced in the community, and suggest future directions.

This monograph offers a survey of work to date to inform how interactions in information retrieval systems could afford inclusion of users who are neurodiverse. This existing work is positioned within a range of philosophies, frameworks and epistemologies which frame the importance of including neurodiverse users in all stages of research and development of Interactive Information Retrieval (IIR) systems. The monograph also offers examples and practical approaches to include neurodiverse users in IIR research, and explores the challenges ahead in the field.

As information retrieval researchers, we not only develop algorithmic solutions to hard problems, but we also insist on a proper, multifaceted evaluation of ideas. The literature on the fundamental topic of retrieval and ranking, for instance, has a rich history of studying the effectiveness of indexes, retrieval algorithms, and complex machine learning rankers, while at the same time quantifying their computational costs, from creation and training to application and inference. This is evidenced, for example, by more than a decade of research on efficient training and inference of large decision forest models in Learning to Rank (LtR). As we move towards even more complex, deep learning models in a wide range of applications, questions on efficiency have once again resurfaced with renewed urgency. Indeed, efficiency is no longer limited to time and space; instead it has found new, challenging dimensions that stretch to resource-, sample- and energy-efficiency with ramifications for researchers, users, and the environment.

This monograph takes a step towards promoting the study of efficiency in the era of neural information retrieval by offering a comprehensive survey of the literature on efficiency and effectiveness in ranking, and to a limited extent, retrieval. This monograph was inspired by the parallels that exist between the challenges in neural network-based ranking solutions and their predecessors, decision forest-based LtR models, as well as the connections between the solutions the literature to date has to offer. We believe that by understanding the fundamentals underpinning these algorithmic and data structure solutions for containing the contentious relationship between efficiency and effectiveness, one can better identify future directions and more efficiently determine the merits of ideas. We also present what we believe to be important research directions in the forefront of efficiency and effectiveness in retrieval and ranking.

The introduction of Quantum Theory (QT) provides a unified mathematical framework for Information Retrieval (IR). Compared with the classical IR framework, the quantuminspired IR framework is based on user-centered modeling methods to model non-classical cognitive phenomena in human relevance judgment in the IR process. With the increase of data and computing resources, neural IR methods have been applied to the text matching and understanding task of IR. Neural networks have a strong learning ability of effective representation and generalization of matching patterns from raw data. However, these methods show some unavoidable defects, such as the inability to model user cognitive phenomena, large number of model parameters and the “black box” characteristics of network structure. These problems greatly limit the development of neural IR and related fields. Although the quantum-inspired retrieval framework can theoretically solve the above problems, it is faced with problems such as poor model efficiency and difficulty in integrating with neural network, which lead to a huge gap between QT and neural network modeling.

This review gives a systematic introduction to quantuminspired neural IR, including quantum-inspired neural language representation, matching and understanding. This is not only helpful to non-classical phenomena modeling in IR but also to break the theoretical bottleneck of neural networks and design more transparent neural IR models. We introduce the language representation method based on QT and the quantum-inspired text matching and decision making model under neural network, which shows its theoretical advantages in document ranking, relevance matching, multimodal IR, and can be integrated with neural networks to jointly promote the development of IR. The latest progress of quantum language understanding is introduced and further topics on QT and language modeling provide readers with more materials for thinking.