Composed image retrieval: a survey on recent research and development

Abstract

In recent years, composed image retrieval (CIR) has gained significant attention within the research community due to its excellent research value and extensive real-world applications. CIR allows modifying query images based on user-provided text descriptions, producing search results that better match users’ intent. This paper conducts a comprehensive and up-to-date survey of CIR research and its applications. We summarise recent advancements in CIR methodologies from these perspectives by breaking down a CIR system into four key processes-feature extraction, feature alignment, feature fusion, and image retrieval. We examine feature extraction, emphasizing deep learning techniques for images and text. As deep learning evolves, feature alignment increasingly integrates with other processes, encouraging us to categorize related methods into explicit and implicit approaches. From the perspective of feature fusion, we investigate advancements in image-text feature fusion techniques, categorizing them into 6 broad categories and 17 subcategories. We also summarize different architecture types and training loss functions for image retrieval. Additionally, we review standard benchmark datasets and evaluation metrics in CIR, presenting a comparative analysis of the accuracy of crucial CIR approaches. Finally, we put forward several critical yet underexplored issues in the field.