Spatio-temporal feature based VLAD for efficient video retrieval

Abstract

Compact representation of visual content has emerged as an important topic in the context of large scale image/video retrieval. The recently proposed Vector of Locally Aggregated Descriptors (VLAD) has shown to outperform other existing techniques for retrieval. In this paper, we propose two spatio-temporal features for constructing VLAD vectors for videos in the context of large scale video retrieval. Given a particular query video, our aim is to retrieve similar videos from the database. Experiments are conducted on UCF50 and HMDB51 datasets, which pose challenges in the form of camera motion, view-point variation, large intra-class variation, etc. The paper proposes the following two spatio-temporal features for constructing VLADs i) Local Histogram of Oriented Optical Flow (LHOOF), and ii) Space-Time Invariant Points (STIP). The performance of these proposed features are compared with SIFT based spatial feature. The mean average precision (MAP) indicates the better retrieval performance of the proposed spatio-temporal feature over spatial feature.