Long-Tailed Object Mining Based on CLIP Model for Autonomous Driving

Abstract

The long-tailed object distribution poses great challenges for autonomous driving. And the field collection of long-tailed objects is difficult and high-cost. In this paper, we propose a novel data mining approach for those long-tailed objects. The softmax distribution produced by CLIP model is adopted as the representation of cropped objects in the image. Then for each long-tailed classification, the category grouping is performed to divide the text concepts into three sets. Finally, combining the softmax representation with the grouped categories, we develop an effective softmax mining algorithm to search and identify the long-tailed objects from the large database. Experiments demonstrate that the proposed method outperforms the baseline results and accurately finds the long-tailed data.