{"title":"情感分析和多模式人机交互的模因检测","authors":"","doi":"10.1109/UPCON56432.2022.9986453","DOIUrl":null,"url":null,"abstract":"Memes are a way of communicating concepts across social media. However, while most memes are intended to be funny, some can turn into offensive as well when text and images are combined together. Recently many successful studies related to sentiment analysis of both image and text have been performed. Such technology, when developed successfully, can be useful for effective Human-Robot-Interactions, specially with humanoid and collaborative robots. In this research, we intend to first develop such technology with available data set using given classes only, since getting labelled data in the robotics domain, specially in robot grasping domain is difficult. In subsequent research, we may extend the same technology for intelligent robot grasping. However, the majority of the research uses either text or images for the sentiment analysis. Since the content and image in memes are sometimes unrelated, detecting hateful memes is a more challenging problem, so the present work considers both as features and uses a multimodal approach for sentiment analysis which could also be useful for Human-Robot-Interactions. Being constrained however with the available data sets, in the present investigation, our focus is on developing multimodal and sequential approaches for classifying these memes into different required classes, more specifically, here two classes: offensive and non-offensive. The fusion approach has been used within multiple modes to take features of both image and text through different models and then it has been used for the classification. While in the sequential approach, the image captioning model which is trained on the MS COCO dataset, with Optical Character Recognition (OCR), is used and classified with the help of the FastText classifier. Both approaches are used on two datasets, one is the MultiOFF dataset, and the other is the Facebook Hateful Meme dataset. Results on both datasets are found to be promising for both approaches.","PeriodicalId":185782,"journal":{"name":"2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Meme Detection For Sentiment Analysis and Human Robot Interactions Using Multiple Modes\",\"authors\":\"\",\"doi\":\"10.1109/UPCON56432.2022.9986453\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Memes are a way of communicating concepts across social media. However, while most memes are intended to be funny, some can turn into offensive as well when text and images are combined together. Recently many successful studies related to sentiment analysis of both image and text have been performed. Such technology, when developed successfully, can be useful for effective Human-Robot-Interactions, specially with humanoid and collaborative robots. In this research, we intend to first develop such technology with available data set using given classes only, since getting labelled data in the robotics domain, specially in robot grasping domain is difficult. In subsequent research, we may extend the same technology for intelligent robot grasping. However, the majority of the research uses either text or images for the sentiment analysis. Since the content and image in memes are sometimes unrelated, detecting hateful memes is a more challenging problem, so the present work considers both as features and uses a multimodal approach for sentiment analysis which could also be useful for Human-Robot-Interactions. Being constrained however with the available data sets, in the present investigation, our focus is on developing multimodal and sequential approaches for classifying these memes into different required classes, more specifically, here two classes: offensive and non-offensive. The fusion approach has been used within multiple modes to take features of both image and text through different models and then it has been used for the classification. While in the sequential approach, the image captioning model which is trained on the MS COCO dataset, with Optical Character Recognition (OCR), is used and classified with the help of the FastText classifier. Both approaches are used on two datasets, one is the MultiOFF dataset, and the other is the Facebook Hateful Meme dataset. Results on both datasets are found to be promising for both approaches.\",\"PeriodicalId\":185782,\"journal\":{\"name\":\"2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)\",\"volume\":\"29 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-12-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/UPCON56432.2022.9986453\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/UPCON56432.2022.9986453","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

表情包是在社交媒体上交流概念的一种方式。然而,虽然大多数表情包都是为了搞笑,但当文字和图片结合在一起时,有些表情包也会变得令人反感。近年来,在图像和文本情感分析方面进行了许多成功的研究。如果开发成功,这种技术可以用于有效的人机交互,特别是与类人机器人和协作机器人。在本研究中,我们打算首先使用仅使用给定类的可用数据集开发这种技术,因为在机器人领域,特别是在机器人抓取领域获得标记数据是困难的。在后续的研究中,我们可以将相同的技术扩展到智能机器人抓取。然而,大多数研究使用文本或图像进行情感分析。由于模因中的内容和图像有时是不相关的,因此检测仇恨模因是一个更具挑战性的问题,因此本研究将两者都视为特征,并使用多模态方法进行情感分析,这也可能对人机交互有用。然而,由于现有数据集的限制,在目前的调查中,我们的重点是开发多模态和顺序方法,将这些模因分类为不同的所需类别,更具体地说,这里分为两类:攻击性和非攻击性。该方法在多种模式下使用,通过不同的模型提取图像和文本的特征,然后将其用于分类。在序列方法中,使用在MS COCO数据集上训练的带有光学字符识别(OCR)的图像字幕模型,并在FastText分类器的帮助下进行分类。这两种方法都用于两个数据集,一个是MultiOFF数据集,另一个是Facebook可恶的Meme数据集。两种方法在两个数据集上的结果都是有希望的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Meme Detection For Sentiment Analysis and Human Robot Interactions Using Multiple Modes
Memes are a way of communicating concepts across social media. However, while most memes are intended to be funny, some can turn into offensive as well when text and images are combined together. Recently many successful studies related to sentiment analysis of both image and text have been performed. Such technology, when developed successfully, can be useful for effective Human-Robot-Interactions, specially with humanoid and collaborative robots. In this research, we intend to first develop such technology with available data set using given classes only, since getting labelled data in the robotics domain, specially in robot grasping domain is difficult. In subsequent research, we may extend the same technology for intelligent robot grasping. However, the majority of the research uses either text or images for the sentiment analysis. Since the content and image in memes are sometimes unrelated, detecting hateful memes is a more challenging problem, so the present work considers both as features and uses a multimodal approach for sentiment analysis which could also be useful for Human-Robot-Interactions. Being constrained however with the available data sets, in the present investigation, our focus is on developing multimodal and sequential approaches for classifying these memes into different required classes, more specifically, here two classes: offensive and non-offensive. The fusion approach has been used within multiple modes to take features of both image and text through different models and then it has been used for the classification. While in the sequential approach, the image captioning model which is trained on the MS COCO dataset, with Optical Character Recognition (OCR), is used and classified with the help of the FastText classifier. Both approaches are used on two datasets, one is the MultiOFF dataset, and the other is the Facebook Hateful Meme dataset. Results on both datasets are found to be promising for both approaches.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Mains Interface Circuit Design for Traveling Wave Tube Amplifier A Passive Technique for Detecting Islanding Using Voltage Sequence Component A Unified Framework for Covariance Adaptation with Multiple Source Domains Advance Sensor for Monitoring Electrolyte Leakage in Lithium-ion Batteries for Electric Vehicles A comparative study of survey papers based on energy efficient, coverage-aware, and fault tolerant in static sink node of WSN
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1