Beyond boundaries: Hierarchical-contrast unsupervised temporal action localization with high-coupling feature learning

Abstract

Current unsupervised temporal action localization (UTAL) methods mainly use clustering and localization with independent learning mechanisms. However, these individual mechanisms are low-coupled and struggle to finely localize action-background boundary information due to the lack of feature interactions in the clustering and localization process. To address this, we propose an end-to-end Hierarchical-Contrast UTAL (HC-UTAL) framework with high-coupling multi-task feature learning. HC-UTAL incorporates coarse-to-fine contrastive learning (CL) at three levels: video level, instance level and boundary level, thus obtaining adaptive interaction and robust performance. We first employ the video-level CL on video-level and cluster-level feature learning, generating video action pseudo-labels. Then, using the video action pseudo-labels, we further devise the instance-level CL on action-related feature learning for coarse localization and the boundary-level CL on ambiguous action-background boundary feature learning for finer localization, respectively. We conduct extensive experiments on THUMOS’14, ActivityNet v1.2, and ActivityNet v1.3 datasets. The results demonstrate that our method achieves state-of-the-art performance. The code and trained models are available at: https://github.com/bugcat9/HC-UTAL.