Video Captioning based on Image Captioning as Subsidiary Content

Abstract

Video captioning is the more heuristic task of the combination of computer vision and Natural language processing while researchers are concentrated more in video related tasks. Dense video captioning is still considering the more challenging task as it needs to consider every event occurs in the video and provide optimal captions separately for all the events presents in the video with high diversity. Captioning process with less corpus leads to less performance. To avoid such issues, our proposed model constructed with the option of generating captions with high diversity. Image captions are taken as subsidiary content to enlarge the diversity for captioning the videos. Attention mechanism is utilized for the generation process. Generator and three different discriminators are utilized to contribute an appropriate caption which enriches the captioning process. ActivityNet caption dataset is used to demonstrate the proposed model. Microsoft coco image dataset is considered as subsidiary content for captioning. The benchmark metrics BLEU and METEOR are used to estimate the performance of the proposed model.