Ontology-Driven Hierarchical Deep Learning for Fashion Recognition

Abstract

We present an automatic approach for large-scale fashion recognition, given an image without any kind of annotation. We formulate the problem as a hierarchical deep learning (HDL) algorithm which can: (i) integrate the deep CNNs to learn more discriminative high-level features for fashion image representations of both coarse-grained and fine-grained classes at different levels of the fashion ontology tree; (ii) leverage multi-task learning and inter-task relationship constraint to train more discriminative classifiers for the nodes on the fashion ontology; (iii) use back propagation to simultaneously refine both the relevant node classifiers and the deep CNNs according to a joint objective function; and (iv) accelerate the fashion retrieval process via path-based classification. The experimental results have verified the effectiveness and efficiency of our proposed algorithm on both classification and retrieval performance.