Bilinear-experts network with self-adaptive sampler for long-tailed visual recognition

Abstract

Long-tail distributed data hinders the practical application of state-of-the-art deep models in computer vision. Consequently, exclusive methodologies for handling the long-tailed problem are proposed, focusing on different hierarchies. For embedding hierarchy, existing works manually augment the diversity of tail-class features for specific datasets. However, prior knowledge about datasets is not always available for practical use, which brings unsatisfactory generalization ability in human fine-turned augmentation under such circumstances. To figure out this problem, we introduce a novel model named Bilinear-Experts Network (BENet) with Self-Adaptive Sampler (SAS). This model leverages model-driven perturbations to tail-class embeddings while preserving generalization capability on head classes through a designed bilinear experts system. The designed perturbations adaptively augment tail-class space and shift the class boundary away from the tail-class centers. Moreover, we find that SAS automatically assigns more significant perturbations to specific tail classes with relatively fewer training samples, which indicates SAS is capable of filtering tail classes with lower quality and enhancing them. Also, experiments conducted across various long-tailed benchmarks validate the comparable performance of the proposed BENet.