A core region captioning framework for automatic video understanding in story video contents

H. Suh, Jiyeon Kim, Jinsoo So, J. Jung
{"title":"A core region captioning framework for automatic video understanding in story video contents","authors":"H. Suh, Jiyeon Kim, Jinsoo So, J. Jung","doi":"10.1177/18479790221078130","DOIUrl":null,"url":null,"abstract":"Due to the rapid increase in images and image data, research examining the visual analysis of such unstructured data has recently come to be actively conducted. One of the representative image caption models the DenseCap model extracts various regions in an image and generates region-level captions. However, since the existing DenseCap model does not consider priority for region captions, it is difficult to identify relatively significant region captions that best describe the image. There has also been a lack of research into captioning focusing on the core areas for story content, such as images in movies and dramas. In this study, we propose a new image captioning framework based on DenseCap that aims to promote the understanding of movies in particular. In addition, we design and implement a module for identifying characters so that the character information can be used in caption detection and caption improvement in core areas. We also propose a core area caption detection algorithm that considers the variables affecting the area caption importance. Finally, a performance evaluation is conducted to determine the accuracy of the character identification module, and the effectiveness of the proposed algorithm is demonstrated by visually comparing it with the existing DenseCap model.","PeriodicalId":45882,"journal":{"name":"International Journal of Engineering Business Management","volume":null,"pages":null},"PeriodicalIF":4.9000,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Engineering Business Management","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1177/18479790221078130","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"BUSINESS","Score":null,"Total":0}
引用次数: 2

Abstract

Due to the rapid increase in images and image data, research examining the visual analysis of such unstructured data has recently come to be actively conducted. One of the representative image caption models the DenseCap model extracts various regions in an image and generates region-level captions. However, since the existing DenseCap model does not consider priority for region captions, it is difficult to identify relatively significant region captions that best describe the image. There has also been a lack of research into captioning focusing on the core areas for story content, such as images in movies and dramas. In this study, we propose a new image captioning framework based on DenseCap that aims to promote the understanding of movies in particular. In addition, we design and implement a module for identifying characters so that the character information can be used in caption detection and caption improvement in core areas. We also propose a core area caption detection algorithm that considers the variables affecting the area caption importance. Finally, a performance evaluation is conducted to determine the accuracy of the character identification module, and the effectiveness of the proposed algorithm is demonstrated by visually comparing it with the existing DenseCap model.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
故事视频内容自动理解的核心区域字幕框架
由于图像和图像数据的快速增长,对这类非结构化数据进行可视化分析的研究最近得到了积极的开展。其中一种代表性的图像标题模型是DenseCap模型,该模型提取图像中的各个区域并生成区域级标题。然而,由于现有的DenseCap模型没有考虑区域标题的优先级,因此很难识别出最能描述图像的相对重要的区域标题。对于故事内容的核心领域,如电影和电视剧中的图像,也缺乏对字幕的研究。在这项研究中,我们提出了一个新的基于DenseCap的图像字幕框架,旨在促进对电影的理解。此外,我们设计并实现了字符识别模块,将字符信息用于核心区域的字幕检测和字幕改进。我们还提出了一种考虑影响区域标题重要性的变量的核心区域标题检测算法。最后,通过性能评估来确定字符识别模块的准确性,并通过与现有的DenseCap模型进行视觉对比来验证所提算法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
7.50
自引率
6.10%
发文量
17
审稿时长
15 weeks
期刊介绍: The International Journal of Engineering Business Management (IJEBM) is an international, peer-reviewed, open access scientific journal that aims to promote an integrated and multidisciplinary approach to engineering, business and management. The journal focuses on issues related to the design, development and implementation of new methodologies and technologies that contribute to strategic and operational improvements of organizations within the contemporary global business environment. IJEBM encourages a systematic and holistic view in order to ensure an integrated and economically, socially and environmentally friendly approach to management of new technologies in business. It aims to be a world-class research platform for academics, managers, and professionals to publish scholarly research in the global arena. All submitted articles considered suitable for the International Journal of Engineering Business Management are subjected to rigorous peer review to ensure the highest levels of quality. The review process is carried out as quickly as possible to minimize any delays in the online publication of articles. Topics of interest include, but are not limited to: -Competitive product design and innovation -Operations and manufacturing strategy -Knowledge management and knowledge innovation -Information and decision support systems -Radio Frequency Identification -Wireless Sensor Networks -Industrial engineering for business improvement -Logistics engineering and transportation -Modeling and simulation of industrial and business systems -Quality management and Six Sigma -Automation of industrial processes and systems -Manufacturing performance and productivity measurement -Supply Chain Management and the virtual enterprise network -Environmental, legal and social aspects -Technology Capital and Financial Modelling -Engineering Economics and Investment Theory -Behavioural, Social and Political factors in Engineering
期刊最新文献
Configurational risks and innovation performance of complex product systems development: A fsQCA lens Entrepreneurship environments and entrepreneurial intention- the role of self efficacy and role model Integrating human factors into the distribution model of goods and fast-moving consumer goods for effective inventory control Towards a framework for successful metaverse adoption in Small and Medium-sized Enterprises: An exploratory study Digital supply chain transformation: The role of smart technologies on operational performance in manufacturing industry
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1