From spatial to semantic: attribute-aware fashion similarity learning via iterative positioning and attribute diverging

Abstract

Fashion image retrieval emphasizes accurately perceiving the fine-grained features to meet users’ precise needs. However, the existing global image-based retrieval methods encounter challenges such as imprecise positioning of attributes, difficulty in distinguishing visually similar but semantically different attribute values, and struggles in the learning of attribute features within specific regions and viewpoints. This paper proposes a two-stage hybrid framework called IPAD (Iterative Positioning and Attribute Diverging) for attribute-aware fashion similarity learning. In the initial stage, we present an iterative positioning strategy to precisely identify local attribute regions through an iterative attention mechanism with adaptive suppression. IPAD leverages the strengths of Convolutional Neural Networks and Vision Transformers. Subsequently, we design an attribute diverging strategy to optimize attribute value aggregation via online clustering using a momentum encoder, thereby enhancing model stability and representation. During inference, we further present a feature reasoning mechanism to refine retrieval results through subgraph similarity matrix generation and re-ranking to enhance accuracy and robustness. Extensive evaluations on three public datasets demonstrate IPAD’s superior performance over state-of-the-art methods in retrieval accuracy, achieving an average improvement in MAP by +4.22%. The source code is available at https://github.com/h8e9r7/IPAD.