基于简单数据扩充的句子表示对比学习算法研究

IF 2.5 4区 综合性期刊 Q2 CHEMISTRY, MULTIDISCIPLINARY Applied Sciences-Basel Pub Date : 2023-09-08 DOI:10.3390/app131810120
Xiaodong Liu, Wenyin Gong, Yuxin Li, Yanchi Li, Xiang Li
{"title":"基于简单数据扩充的句子表示对比学习算法研究","authors":"Xiaodong Liu, Wenyin Gong, Yuxin Li, Yanchi Li, Xiang Li","doi":"10.3390/app131810120","DOIUrl":null,"url":null,"abstract":"In the era of deep learning, representational text-matching algorithms based on BERT and its variant models have become mainstream and are limited by the sentence vectors generated by the BERT model, and the SimCSE algorithm proposed in 2021 has improved the sentence vector quality to a certain extent. In this paper, to address the problem that the SimCSE algorithm has—that the greater the difference in sentence length, the smaller the probability that the sentence pairs are similar—an EdaCSE algorithm is proposed to perturb the sentence length using a simple data enhancement method without affecting the semantics of the sentences. The perturbation is applied to the sentence length by adding meaningless English punctuation marks to the original sentence so that the model no longer tends to recognise sentences of similar length as similar sentences. Based on the BERT series of models, experiments were conducted on five different datasets, and the experiments proved that the EdaCSE method improves an average of 1.67, 0.84, and 1.08 on the five datasets.","PeriodicalId":48760,"journal":{"name":"Applied Sciences-Basel","volume":null,"pages":null},"PeriodicalIF":2.5000,"publicationDate":"2023-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Study of Contrastive Learning Algorithms for Sentence Representation Based on Simple Data Augmentation\",\"authors\":\"Xiaodong Liu, Wenyin Gong, Yuxin Li, Yanchi Li, Xiang Li\",\"doi\":\"10.3390/app131810120\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In the era of deep learning, representational text-matching algorithms based on BERT and its variant models have become mainstream and are limited by the sentence vectors generated by the BERT model, and the SimCSE algorithm proposed in 2021 has improved the sentence vector quality to a certain extent. In this paper, to address the problem that the SimCSE algorithm has—that the greater the difference in sentence length, the smaller the probability that the sentence pairs are similar—an EdaCSE algorithm is proposed to perturb the sentence length using a simple data enhancement method without affecting the semantics of the sentences. The perturbation is applied to the sentence length by adding meaningless English punctuation marks to the original sentence so that the model no longer tends to recognise sentences of similar length as similar sentences. Based on the BERT series of models, experiments were conducted on five different datasets, and the experiments proved that the EdaCSE method improves an average of 1.67, 0.84, and 1.08 on the five datasets.\",\"PeriodicalId\":48760,\"journal\":{\"name\":\"Applied Sciences-Basel\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.5000,\"publicationDate\":\"2023-09-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Applied Sciences-Basel\",\"FirstCategoryId\":\"103\",\"ListUrlMain\":\"https://doi.org/10.3390/app131810120\",\"RegionNum\":4,\"RegionCategory\":\"综合性期刊\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"CHEMISTRY, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Sciences-Basel","FirstCategoryId":"103","ListUrlMain":"https://doi.org/10.3390/app131810120","RegionNum":4,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CHEMISTRY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

摘要

在深度学习时代,基于BERT及其变体模型的代表性文本匹配算法已成为主流,并受到BERT模型生成的句子向量的限制,2021年提出的SimCSE算法在一定程度上提高了句子向量质量。在本文中,为了解决SimCSE算法存在的问题——句子长度的差异越大,句子对相似的概率越小——提出了一种EdaCSE算法,在不影响句子语义的情况下,使用简单的数据增强方法来干扰句子长度。通过在原始句子中添加无意义的英语标点符号,将扰动应用于句子长度,使模型不再倾向于将长度相似的句子识别为相似的句子。基于BERT系列模型,在五个不同的数据集上进行了实验,实验证明EdaCSE方法在五个数据集上平均提高了1.67、0.84和1.08。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
A Study of Contrastive Learning Algorithms for Sentence Representation Based on Simple Data Augmentation
In the era of deep learning, representational text-matching algorithms based on BERT and its variant models have become mainstream and are limited by the sentence vectors generated by the BERT model, and the SimCSE algorithm proposed in 2021 has improved the sentence vector quality to a certain extent. In this paper, to address the problem that the SimCSE algorithm has—that the greater the difference in sentence length, the smaller the probability that the sentence pairs are similar—an EdaCSE algorithm is proposed to perturb the sentence length using a simple data enhancement method without affecting the semantics of the sentences. The perturbation is applied to the sentence length by adding meaningless English punctuation marks to the original sentence so that the model no longer tends to recognise sentences of similar length as similar sentences. Based on the BERT series of models, experiments were conducted on five different datasets, and the experiments proved that the EdaCSE method improves an average of 1.67, 0.84, and 1.08 on the five datasets.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Applied Sciences-Basel
Applied Sciences-Basel CHEMISTRY, MULTIDISCIPLINARYMATERIALS SCIE-MATERIALS SCIENCE, MULTIDISCIPLINARY
CiteScore
5.30
自引率
11.10%
发文量
10882
期刊介绍: Applied Sciences (ISSN 2076-3417) provides an advanced forum on all aspects of applied natural sciences. It publishes reviews, research papers and communications. Our aim is to encourage scientists to publish their experimental and theoretical results in as much detail as possible. There is no restriction on the length of the papers. The full experimental details must be provided so that the results can be reproduced. Electronic files and software regarding the full details of the calculation or experimental procedure, if unable to be published in a normal way, can be deposited as supplementary electronic material.
期刊最新文献
Application of Digital Holographic Imaging to Monitor Real-Time Cardiomyocyte Hypertrophy Dynamics in Response to Norepinephrine Stimulation. Reproducibility in Radiomics: A Comparison of Feature Extraction Methods and Two Independent Datasets. Study on Shear Resistance and Structural Performance of Corrugated Steel–Concrete Composite Deck Clustering Analysis of Wind Turbine Alarm Sequences Based on Domain Knowledge-Fused Word2vec Unraveling Functional Dysphagia: A Game-Changing Automated Machine-Learning Diagnostic Approach
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1