{"title":"A core region captioning framework for automatic video understanding in story video contents","authors":"H. Suh, Jiyeon Kim, Jinsoo So, J. Jung","doi":"10.1177/18479790221078130","DOIUrl":null,"url":null,"abstract":"Due to the rapid increase in images and image data, research examining the visual analysis of such unstructured data has recently come to be actively conducted. One of the representative image caption models the DenseCap model extracts various regions in an image and generates region-level captions. However, since the existing DenseCap model does not consider priority for region captions, it is difficult to identify relatively significant region captions that best describe the image. There has also been a lack of research into captioning focusing on the core areas for story content, such as images in movies and dramas. In this study, we propose a new image captioning framework based on DenseCap that aims to promote the understanding of movies in particular. In addition, we design and implement a module for identifying characters so that the character information can be used in caption detection and caption improvement in core areas. We also propose a core area caption detection algorithm that considers the variables affecting the area caption importance. Finally, a performance evaluation is conducted to determine the accuracy of the character identification module, and the effectiveness of the proposed algorithm is demonstrated by visually comparing it with the existing DenseCap model.","PeriodicalId":45882,"journal":{"name":"International Journal of Engineering Business Management","volume":null,"pages":null},"PeriodicalIF":4.9000,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Engineering Business Management","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1177/18479790221078130","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"BUSINESS","Score":null,"Total":0}
引用次数: 2
Abstract
Due to the rapid increase in images and image data, research examining the visual analysis of such unstructured data has recently come to be actively conducted. One of the representative image caption models the DenseCap model extracts various regions in an image and generates region-level captions. However, since the existing DenseCap model does not consider priority for region captions, it is difficult to identify relatively significant region captions that best describe the image. There has also been a lack of research into captioning focusing on the core areas for story content, such as images in movies and dramas. In this study, we propose a new image captioning framework based on DenseCap that aims to promote the understanding of movies in particular. In addition, we design and implement a module for identifying characters so that the character information can be used in caption detection and caption improvement in core areas. We also propose a core area caption detection algorithm that considers the variables affecting the area caption importance. Finally, a performance evaluation is conducted to determine the accuracy of the character identification module, and the effectiveness of the proposed algorithm is demonstrated by visually comparing it with the existing DenseCap model.
期刊介绍:
The International Journal of Engineering Business Management (IJEBM) is an international, peer-reviewed, open access scientific journal that aims to promote an integrated and multidisciplinary approach to engineering, business and management. The journal focuses on issues related to the design, development and implementation of new methodologies and technologies that contribute to strategic and operational improvements of organizations within the contemporary global business environment. IJEBM encourages a systematic and holistic view in order to ensure an integrated and economically, socially and environmentally friendly approach to management of new technologies in business. It aims to be a world-class research platform for academics, managers, and professionals to publish scholarly research in the global arena. All submitted articles considered suitable for the International Journal of Engineering Business Management are subjected to rigorous peer review to ensure the highest levels of quality. The review process is carried out as quickly as possible to minimize any delays in the online publication of articles. Topics of interest include, but are not limited to: -Competitive product design and innovation -Operations and manufacturing strategy -Knowledge management and knowledge innovation -Information and decision support systems -Radio Frequency Identification -Wireless Sensor Networks -Industrial engineering for business improvement -Logistics engineering and transportation -Modeling and simulation of industrial and business systems -Quality management and Six Sigma -Automation of industrial processes and systems -Manufacturing performance and productivity measurement -Supply Chain Management and the virtual enterprise network -Environmental, legal and social aspects -Technology Capital and Financial Modelling -Engineering Economics and Investment Theory -Behavioural, Social and Political factors in Engineering