An Image Caption Model Based on the Scene Graph and Semantic Prior Network

Abstract

In this paper, we propose an image caption model based on scene graphs and semantic priors to address the problem of traditional image caption models that are overly dependent on training data. First, the original image features and the scene graph features are fused by embedding the scene image features into the feature space. Second, using image captions from the existing dataset, the sentence reconstruction task is used to train the memory network to retain semantic prior knowledge. The scene graph features are then combined with semantic prior information to reconstruct the new features, which are then sent into the Decoder to produce an image caption.