Learning differentiable categorical regions with Gumbel-Softmax for person re-identification

Abstract

Locating diverse body parts and perceiving part visibility are essential to person re-identification (re-ID). Most existing methods employ an extra model, e.g., pose estimation or human parsing, to locate parts, or generate pseudo labels to train the part locator incorporated with the re-ID model. In this paper, we aim at learning diverse horizontal stripes with foreground refinement to pursue pixel-level part alignment via only using person identity labels. Specifically, we proposed a Gumbel-Softmax based Differential Categorical Region (DCR) learning method and make two contributions. (1) A stripe-wise regularization. Given an image, the part locator produce part probability maps. The continuous values in the probability maps are discretized into zero or $\arg \max$ value in the horizontal stripes by the Gumbel-Softmax. Gumbel-Softmax allows us to use the $\arg \max$ discrete value for part diversity regularization in the forward pass, but can still estimate gradients in the backward pass. (2) A self-refinement method to suppress the background noise in the stripes. We employ a lightweight foreground perception head to produce foreground probability map with only person identity labels supervision. Benefits from discretization of the categorical stripes, we can conveniently obtain the part pseudo label by element-wise multiplying the categorical stripes with foreground probability map. Finally, DCR can locate the body parts at pixel-level and extract part-aligned representation. Experimental results on both holistic and occluded re-ID datasets confirm that our approach significantly improves the learned representation and the achieved performance is on par with the state-of-the-art methods. The code is available at https://github.com/deepalchemist/differentiable-categorical-region