Scene-enhanced multi-scale temporal aware network for video moment retrieval

Abstract

Video moment retrieval aims to locate the target moment in an untrimmed video using a natural language query. Current methods to moment retrieval are typically tailored for scenarios where temporal localization information is often simple. Nevertheless, these methods overlook the scenarios where a video includes complex localization information, which makes it difficult to achieve precise retrieval across videos that encompass both complex and simple temporal localization information. To address this limitation, we propose a novel Scene-enhanced Multi-scale Temporal Aware Network (SMTAN) designed to adaptively extract different temporal localization information in different videos. Our method involves the comprehensive processing of video moments across fine-grained multiply scales and uses a prior knowledge of the scene for localization information enhancement. This method facilitates the construction of multi-scale temporal feature maps, enabling extraction of both complex and simple temporal localization information in different videos. Extensive experiments on two benchmark datasets demonstrate that our proposed network surpasses the state-of-the-art methods and achieves more accurate retrieval of different localization information across videos.