CSA-BERT: Video Question Answering

Abstract

Convolutional networks are a key component of many computer vision applications. However, convolutions have a serious flaw. It only works in a small area, hence it lacks global information. The Attention method, on the other hand, is a new improvement in capturing long range interactions that has mostly been used to sequence modeling and generative modeling tasks. As an alternative to convolutions, we investigate the use of convolutions with an attention mechanism in a video question answering task. We present a unique self-attention mechanism based on convolutions that outperforms convolutions in the video question answering task. We discovered that combining convolutions with self-attention produces the greatest outcomes in experiments. As a result, we propose a hybrid idea, which combines convolutional operators with the self-attention mechanism. We combine convolutional feature maps with self-attention feature maps. Experiments show that convolution with self-attention improves video question answering tasks on the MSRVTT-QA dataset.