Sparse-Guided Partial Dense for Cross-Modal Remote Sensing Image–Text Retrieval

Abstract

Cross-modal remote sensing image-text retrieval (CMRSITR) involves retrieving relevant samples in one modality based on a query from another modality. Previous dense retrieval methods utilizing multivector dense representations have significantly enhanced retrieval performance. Meanwhile, recent advances in sparse retrieval have demonstrated that sparse representations offer comparable performance with enhanced interpretability and faster retrieval speeds. However, effectively integrating the strengths of these two paradigms to enable efficient and accurate retrieval in large-scale remote sensing (RS) image-text datasets remains an open challenge. In this study, we propose sparse-guided partial dense (SGPD) cross-modal retrieval, a novel approach that efficiently transforms dense vectors from pretrained dense retrieval models into sparse representations and leverages the overlap between sparse retrieval results and dense vector clusters to achieve high-precision and fast retrieval. By probabilistically selecting a limited number of dense clusters containing top sparse results, SGPD ensures retrieval efficiency while minimizing both memory and time costs. Designed as a plug-and-play solution, SGPD can be seamlessly integrated into existing RS image-text retrieval (RSITR) models without requiring modifications to their architectures. Extensive experiments on RS image-text datasets of varying scales demonstrate that SGPD achieves retrieval accuracy comparable to dense retrieval methods while significantly reducing training time and memory consumption.