Prior-Structure Driven Weakly-Supervised Learning for Fine-Grained Human Parsing

Abstract

Weakly-supervised fine-grained human parsing, which decomposes the human body into several parts and various fashion items only with some easier labels, poses a more challenging visual task and cannot be well solved by general weakly-supervised approaches. In this case, we first explore the feasibility of utilizing point-level labels to address this task. Toward this, we propose the prior-structure driven weakly-supervised learning for fine-grained human parsing. Following previous practices, we design a pseudo label initialization mechanism to produce high-quality pixel-level pseudo labels by utilizing the powerful image segmentation model Segment Anything Model (SAM). Then we propose the Feature Propagation based on Prior-Structure (FPPS) module which formalizes prior-structure knowledge as an adjacency matrix constructed from superpixel and emploies a learnable Graph Neural Network (GNN) as the feature propagator. FPPS can optimize the features of unlabeled pixels to enhance the weakly-supervised learning. The framework further designs the Refinement Pseudo Label (RPL) strategy to generate denser supervision from past sub-optimal models. To the best knowledge, this work is the first attempt to perform fine-grained human parsing in a weakly-supervised manner. We conduct extensive experiments on two challenging fine-grained datasets, including ATR and LIP. Experimental results show that the proposed weakly-supervised method yields a comparable result to strongly-supervised methods and even outperforms other state-of-the-art approaches in semi-supervised human parsing tasks.