Scene Retrieval in Soccer Videos by Spatial-temporal Attention with Video Vision Transformer

Abstract

This paper presents a scene retrieval method in soccer videos with video vision Transformer (ViViT). In soccer coaching, it is difficult for the training staff to find the required scenes efficiently from the large number of soccer videos. We tackle this problem with a simple yet effective method. We train ViViT and obtain the output token features of the soccer scene by the pre-trained ViViT model. The output tokens of the pre-trained ViViT contain spatio-temporal information of soccer scenes. We then transform a query scene and candidate scenes into output token features using the pre-trained ViViT and calculate the similarity between the tokens with cosine similarity. We conducted experiments on SoccerNet-V2dataset. The experimental results show that the proposed method achieves outstanding retrieval accuracy compared to the previous methods.