Self-Assembled Generative Framework for Generalized Zero-Shot Learning

Abstract

Generative models have attracted much attention for handling the generalized zero-shot learning (GZSL) task recently. Most of the existing generative GZSL models are trained for visual feature synthesis by utilizing the unique semantic feature of each object class as input but its kaleidoscopic real visual features as supervisions. However, since the real visual features are inevitably infiltrated by some class-irrelevant information, the trained generative models could not guarantee the discriminability of their synthesized visual features. In this paper, we firstly provide an empirical analysis on this problem, finding that among the elements of the real visual features, some elements contain more class-irrelevant information than the others, resulting in ambiguous visual feature synthesis. Then according to this finding, we propose a self-assembled generative GZSL framework, where both the real and synthesized visual features are re-assembled by identifying and updating the class-irrelevant elements in a self-learning manner, called SaG. Moreover, an element-affinity regularizer is explored for constraining the affinity among different elements, so that the synthesized visual features under the SaG framework approach the updated feature elements. In principle, different generative GZSL models could be seamlessly embedded into the SaG framework, resulting in different GZSL methods. Extensive experimental results demonstrate that the derived methods, by embedding three baseline generative GZSL models into SaG respectively, could boost the performances of their baselines significantly, and one of the derived methods outperforms 20 state-of-the-art GZSL methods in most cases.