Combinatorial action recognition based on causal segment intervention

Abstract

Combinatorial action recognition has recently attracted the attention of researchers in the field of computer vision. It focuses on the effective representation and discrimination of spatio-temporal interactions occurring between different actions and objects in video data. Existing work tends to strengthen the framework's object recognition capabilities and relationship modeling capabilities, e.g., attention mechanisms, and graph structures. We find that existing algorithms can be influenced by interaction-independent video segments in a video, misleading the algorithm to focus on additional information in the vision. For the algorithm to analyze the spatio-temporal interactions of causally related video segments in a video, a Causal Slice Recognition Network (CSRN) is proposed. This method can effectively remove the interference of video background segments by explicitly recognizing and extracting the causally related segments in the video. We validate the method on the Something-else dataset and obtain the best results.