首页 > 最新文献

Proceedings of the conference. Association for Computational Linguistics. Meeting最新文献

英文 中文
Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment. 使用词级对齐的层次注意策略进行多模式情感分析。
Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, Ivan Marsic

Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still challenging because:(i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model's synchronized attention over modalities offers visual interpretability.

多模式情感计算,学习识别和解释来自多个数据源的人类情感和主观信息,仍然具有挑战性,因为:(i)很难从异质输入中提取信息特征来表示人类情感;(ii)目前的融合策略只在抽象层面融合不同的模态,忽略了模态之间与时间相关的相互作用。针对这些问题,我们引入了一种具有注意力和词级融合的分层多模式架构,以从文本和音频数据中对话语级情感和情绪进行分类。我们引入的模型在已发布的数据集上优于最先进的方法,我们证明了我们的模型对模态的同步关注提供了视觉可解释性。
{"title":"Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment.","authors":"Yue Gu,&nbsp;Kangning Yang,&nbsp;Shiyu Fu,&nbsp;Shuhong Chen,&nbsp;Xinyu Li,&nbsp;Ivan Marsic","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still challenging because:(i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model's synchronized attention over modalities offers visual interpretability.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2018 ","pages":"2225-2235"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6261375/pdf/nihms-993286.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41174352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Compositional Language Modeling for Icon-Based Augmentative and Alternative Communication. 基于图标的组合语言建模的辅助和替代交流。
Shiran Dudy, Steven Bedrick

Icon-based communication systems are widely used in the field of Augmentative and Alternative Communication. Typically, icon-based systems have lagged behind word- and character-based systems in terms of predictive typing functionality, due to the challenges inherent to training icon-based language models. We propose a method for synthesizing training data for use in icon-based language models, and explore two different modeling strategies.

基于图标的通信系统在辅助通信和替代通信领域有着广泛的应用。通常,由于训练基于图标的语言模型所固有的挑战,基于图标的系统在预测键入功能方面落后于基于单词和字符的系统。我们提出了一种用于基于图标的语言模型的综合训练数据的方法,并探索了两种不同的建模策略。
{"title":"Compositional Language Modeling for Icon-Based Augmentative and Alternative Communication.","authors":"Shiran Dudy,&nbsp;Steven Bedrick","doi":"10.18653/v1/w18-3404","DOIUrl":"https://doi.org/10.18653/v1/w18-3404","url":null,"abstract":"<p><p>Icon-based communication systems are widely used in the field of Augmentative and Alternative Communication. Typically, icon-based systems have lagged behind word- and character-based systems in terms of predictive typing functionality, due to the challenges inherent to training icon-based language models. We propose a method for synthesizing training data for use in icon-based language models, and explore two different modeling strategies.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2018 ","pages":"25-32"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8087438/pdf/nihms-1001619.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"38939330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
Training Classifiers with Natural Language Explanations. 用自然语言解释训练分类器
Braden Hancock, Martin Bringmann, Paroma Varma, Percy Liang, Stephanie Wang, Christopher Ré

Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100× faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.

训练精确的分类器需要许多标签,但每个标签只能提供有限的信息(二进制分类时为一个比特)。在这项工作中,我们提出了一个用于训练分类器的框架--BabbleLabble,在这个框架中,注释者为每个标签决定提供自然语言解释。语义解析器将这些解释转换成程序化的标签函数,为任意数量的未标签数据生成噪声标签,用于训练分类器。在三个关系提取任务中,我们发现用户通过提供解释而不仅仅是标签,能够以 5-100 倍的速度训练分类器,并获得相当的 F1 分数。此外,考虑到标签功能本身的不完善,我们发现基于规则的简单语义解析器就足够了。
{"title":"Training Classifiers with Natural Language Explanations.","authors":"Braden Hancock, Martin Bringmann, Paroma Varma, Percy Liang, Stephanie Wang, Christopher Ré","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100× faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2018 ","pages":"1884-1895"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6534135/pdf/nihms-993798.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"37013648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature. 一个包含患者、干预措施和结果的多层次注释的语料库,以支持医学文献的语言处理。
Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain J Marshall, Ani Nenkova, Byron C Wallace

We present a corpus of 5,000 richly annotated abstracts of medical articles describing clinical randomized controlled trials. Annotations include demarcations of text spans that describe the Patient population enrolled, the Interventions studied and to what they were Compared, and the Outcomes measured (the 'PICO' elements). These spans are further annotated at a more granular level, e.g., individual interventions within them are marked and mapped onto a structured medical vocabulary. We acquired annotations from a diverse set of workers with varying levels of expertise and cost. We describe our data collection process and the corpus itself in detail. We then outline a set of challenging NLP tasks that would aid searching of the medical literature and the practice of evidence-based medicine.

我们提供了一个由5000篇注释丰富的医学文章摘要组成的语料库,这些文章描述了临床随机对照试验。注释包括描述入选患者群体、所研究的干预措施和与之比较的文本跨度的分界,以及测量的结果(“PICO”元素)。这些跨度在更精细的层面上被进一步注释,例如,其中的个体干预被标记并映射到结构化的医学词汇表中。我们从具有不同专业知识和成本水平的不同员工那里获得了注释。我们详细描述了我们的数据收集过程和语料库本身。然后,我们概述了一系列具有挑战性的NLP任务,这些任务将有助于检索医学文献和循证医学实践。
{"title":"A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature.","authors":"Benjamin Nye,&nbsp;Junyi Jessy Li,&nbsp;Roma Patel,&nbsp;Yinfei Yang,&nbsp;Iain J Marshall,&nbsp;Ani Nenkova,&nbsp;Byron C Wallace","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>We present a corpus of 5,000 richly annotated abstracts of medical articles describing clinical randomized controlled trials. Annotations include demarcations of text spans that describe the Patient population enrolled, the Interventions studied and to what they were Compared, and the Outcomes measured (the 'PICO' elements). These spans are further annotated at a more granular level, e.g., individual interventions within them are marked and mapped onto a structured medical vocabulary. We acquired annotations from a diverse set of workers with varying levels of expertise and cost. We describe our data collection process and the corpus itself in detail. We then outline a set of challenging NLP tasks that would aid searching of the medical literature and the practice of evidence-based medicine.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2018 ","pages":"197-207"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6174533/pdf/nihms-988059.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41158894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature 支持医学文献语言处理的病人、干预和结果多层次注释语料库
Benjamin E. Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, I. Marshall, A. Nenkova, Byron C. Wallace
We present a corpus of 5,000 richly annotated abstracts of medical articles describing clinical randomized controlled trials. Annotations include demarcations of text spans that describe the Patient population enrolled, the Interventions studied and to what they were Compared, and the Outcomes measured (the ‘PICO’ elements). These spans are further annotated at a more granular level, e.g., individual interventions within them are marked and mapped onto a structured medical vocabulary. We acquired annotations from a diverse set of workers with varying levels of expertise and cost. We describe our data collection process and the corpus itself in detail. We then outline a set of challenging NLP tasks that would aid searching of the medical literature and the practice of evidence-based medicine.
我们提出了5000篇医学文章的丰富注释摘要的语料库,这些文章描述了临床随机对照试验。注释包括描述入组患者人群、研究干预措施及其比较的文本范围的划分,以及测量的结果(“PICO”元素)。这些跨度在更细粒度的层次上被进一步注释,例如,其中的个体干预被标记并映射到结构化的医学词汇中。我们从具有不同水平的专业知识和成本的不同工作人员那里获得注释。我们详细描述了我们的数据收集过程和语料库本身。然后,我们概述了一组具有挑战性的NLP任务,这些任务将有助于搜索医学文献和循证医学的实践。
{"title":"A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature","authors":"Benjamin E. Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, I. Marshall, A. Nenkova, Byron C. Wallace","doi":"10.18653/v1/P18-1019","DOIUrl":"https://doi.org/10.18653/v1/P18-1019","url":null,"abstract":"We present a corpus of 5,000 richly annotated abstracts of medical articles describing clinical randomized controlled trials. Annotations include demarcations of text spans that describe the Patient population enrolled, the Interventions studied and to what they were Compared, and the Outcomes measured (the ‘PICO’ elements). These spans are further annotated at a more granular level, e.g., individual interventions within them are marked and mapped onto a structured medical vocabulary. We acquired annotations from a diverse set of workers with varying levels of expertise and cost. We describe our data collection process and the corpus itself in detail. We then outline a set of challenging NLP tasks that would aid searching of the medical literature and the practice of evidence-based medicine.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"120 1","pages":"197-207"},"PeriodicalIF":0.0,"publicationDate":"2018-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76655771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 162
Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment 基于层次注意策略的多模态情感分析
Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, I. Marsic
Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still a challenge because: (i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model is able to visualize and interpret synchronized attention over modalities.
学习识别和解释来自多个数据源的人类情感和主观信息的多模态情感计算仍然是一个挑战,因为:(i)很难从异构输入中提取信息特征来表示人类情感;(ii)当前的融合策略仅在抽象层面上融合不同的模式,忽略了模式之间依赖于时间的相互作用。为了解决这些问题,我们引入了一种具有注意力和词级融合的分层多模态架构来对文本和音频数据中的话语级情感和情感进行分类。我们引入的模型在已发布的数据集上优于最先进的方法,并且我们证明了我们的模型能够可视化和解释模态上的同步注意力。
{"title":"Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment","authors":"Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, I. Marsic","doi":"10.18653/v1/P18-1207","DOIUrl":"https://doi.org/10.18653/v1/P18-1207","url":null,"abstract":"Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still a challenge because: (i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model is able to visualize and interpret synchronized attention over modalities.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"68 1","pages":"2225-2235"},"PeriodicalIF":0.0,"publicationDate":"2018-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79113531","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 100
Training Classifiers with Natural Language Explanations 用自然语言解释训练分类器
Braden Hancock, P. Varma, Stephanie Wang, Martin Bringmann, Percy Liang, C. Ré
Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100 faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.
训练准确的分类器需要许多标签,但是每个标签只提供有限的信息(对于二值分类来说是一个比特)。在这项工作中,我们提出了BabbleLabble,这是一个用于训练分类器的框架,其中注释器为每个标记决策提供自然语言解释。语义解析器将这些解释转换为可编程标记函数,该函数为任意数量的未标记数据生成嘈杂的标签,用于训练分类器。在三个关系提取任务中,我们发现用户能够通过提供解释而不仅仅是标签来更快地训练具有5-100分可比较F1分数的分类器。此外,考虑到标记功能的固有缺陷,我们发现一个简单的基于规则的语义解析器就足够了。
{"title":"Training Classifiers with Natural Language Explanations","authors":"Braden Hancock, P. Varma, Stephanie Wang, Martin Bringmann, Percy Liang, C. Ré","doi":"10.18653/v1/P18-1175","DOIUrl":"https://doi.org/10.18653/v1/P18-1175","url":null,"abstract":"Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100 faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2 1","pages":"1884-1895"},"PeriodicalIF":0.0,"publicationDate":"2018-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83375273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 140
Automating Biomedical Evidence Synthesis: RobotReviewer. 自动化生物医学证据合成:RobotReviewer。
Iain J Marshall, Joël Kuiper, Edward Banner, Byron C Wallace

We present RobotReviewer, an open-source web-based system that uses machine learning and NLP to semi-automate biomedical evidence synthesis, to aid the practice of Evidence-Based Medicine. RobotReviewer processes full-text journal articles (PDFs) describing randomized controlled trials (RCTs). It appraises the reliability of RCTs and extracts text describing key trial characteristics (e.g., descriptions of the population) using novel NLP methods. RobotReviewer then automatically generates a report synthesising this information. Our goal is for RobotReviewer to automatically extract and synthesise the full-range of structured data needed to inform evidence-based practice.

我们介绍了RobotReviewer,一个基于web的开源系统,它使用机器学习和NLP来半自动生物医学证据合成,以帮助循证医学的实践。RobotReviewer处理描述随机对照试验(rct)的全文期刊文章(pdf)。它评估随机对照试验的可靠性,并使用新颖的NLP方法提取描述关键试验特征(例如,对人群的描述)的文本。然后RobotReviewer自动合成这些信息生成一份报告。我们的目标是让RobotReviewer自动提取和合成所需的全方位结构化数据,为循证实践提供信息。
{"title":"Automating Biomedical Evidence Synthesis: RobotReviewer.","authors":"Iain J Marshall, Joël Kuiper, Edward Banner, Byron C Wallace","doi":"10.18653/v1/P17-4002","DOIUrl":"10.18653/v1/P17-4002","url":null,"abstract":"<p><p>We present <i>RobotReviewer</i>, an open-source web-based system that uses machine learning and NLP to semi-automate biomedical evidence synthesis, to aid the practice of Evidence-Based Medicine. RobotReviewer processes full-text journal articles (PDFs) describing randomized controlled trials (RCTs). It appraises the reliability of RCTs and extracts text describing key trial characteristics (e.g., descriptions of the population) using novel NLP methods. RobotReviewer then automatically generates a report synthesising this information. Our goal is for RobotReviewer to automatically extract and synthesise the full-range of structured data needed to inform evidence-based practice.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2017 ","pages":"7-12"},"PeriodicalIF":0.0,"publicationDate":"2017-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5662138/pdf/nihms880383.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35568129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 54
Neural Semantic Encoders. 神经语义编码器。
Tsendsuren Munkhdalai, Hong Yu

We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read, compose and write operations. NSE can also access multiple and shared memories. In this paper, we demonstrated the effectiveness and the flexibility of NSE on five different natural language tasks: natural language inference, question answering, sentence classification, document sentiment analysis and machine translation where NSE achieved state-of-the-art performance when evaluated on publically available benchmarks. For example, our shared-memory model showed an encouraging result on neural machine translation, improving an attention-based baseline by approximately 1.0 BLEU.

我们提出了一种用于自然语言理解的记忆增强神经网络:神经语义编码器。NSE配备了一种新颖的内存更新规则,并具有可变大小的编码内存,随着时间的推移而发展,并通过读、写和写操作保持对输入序列的理解。NSE还可以访问多个内存和共享内存。在本文中,我们展示了NSE在五个不同的自然语言任务上的有效性和灵活性:自然语言推理、问题回答、句子分类、文档情感分析和机器翻译,其中NSE在公开可用的基准测试中获得了最先进的性能。例如,我们的共享内存模型在神经机器翻译上显示出令人鼓舞的结果,将基于注意力的基线提高了大约1.0 BLEU。
{"title":"Neural Semantic Encoders.","authors":"Tsendsuren Munkhdalai,&nbsp;Hong Yu","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through <i>read</i>, <i>compose</i> and <i>write</i> operations. NSE can also access multiple and shared memories. In this paper, we demonstrated the effectiveness and the flexibility of NSE on five different natural language tasks: natural language inference, question answering, sentence classification, document sentiment analysis and machine translation where NSE achieved state-of-the-art performance when evaluated on publically available benchmarks. For example, our shared-memory model showed an encouraging result on neural machine translation, improving an attention-based baseline by approximately 1.0 BLEU.</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"1 ","pages":"397-407"},"PeriodicalIF":0.0,"publicationDate":"2017-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5657452/pdf/nihms912000.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35553901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Neural Tree Indexers for Text Understanding. 文本理解的神经树索引器。
Tsendsuren Munkhdalai, Hong Yu

Recurrent neural networks (RNNs) process input text sequentially and model the conditional transition between word tokens. In contrast, the advantages of recursive networks include that they explicitly model the compositionality and the recursive structure of natural language. However, the current recursive architecture is limited by its dependence on syntactic tree. In this paper, we introduce a robust syntactic parsing-independent tree structured model, Neural Tree Indexers (NTI) that provides a middle ground between the sequential RNNs and the syntactic tree-based recursive models. NTI constructs a full n-ary tree by processing the input text with its node function in a bottom-up fashion. Attention mechanism can then be applied to both structure and node function. We implemented and evaluated a binary-tree model of NTI, showing the model achieved the state-of-the-art performance on three different NLP tasks: natural language inference, answer sentence selection, and sentence classification, outperforming state-of-the-art recurrent and recursive neural networks .

递归神经网络(RNNs)对输入文本进行顺序处理,并对单词标记之间的条件转换进行建模。相比之下,递归网络的优点包括它们明确地模拟了自然语言的组合性和递归结构。然而,目前的递归体系结构受其对语法树的依赖的限制。在本文中,我们介绍了一个鲁棒的独立于语法解析的树结构模型,神经树索引器(NTI),它提供了一个介于顺序rnn和基于语法树的递归模型之间的中间地带。NTI通过使用其节点函数以自下而上的方式处理输入文本来构造一个完整的n元树。注意机制可以同时应用于结构和节点功能。我们实现并评估了NTI的二叉树模型,表明该模型在三个不同的NLP任务上取得了最先进的性能:自然语言推理、答案句子选择和句子分类,优于最先进的循环和递归神经网络。
{"title":"Neural Tree Indexers for Text Understanding.","authors":"Tsendsuren Munkhdalai,&nbsp;Hong Yu","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Recurrent neural networks (RNNs) process input text sequentially and model the conditional transition between word tokens. In contrast, the advantages of recursive networks include that they explicitly model the compositionality and the recursive structure of natural language. However, the current recursive architecture is limited by its dependence on syntactic tree. In this paper, we introduce a robust syntactic parsing-independent tree structured model, Neural Tree Indexers (NTI) that provides a middle ground between the sequential RNNs and the syntactic tree-based recursive models. NTI constructs a <i>full n-ary tree</i> by processing the input text with its node function in a bottom-up fashion. Attention mechanism can then be applied to both structure and node function. We implemented and evaluated a binary-tree model of NTI, showing the model achieved the state-of-the-art performance on three different NLP tasks: natural language inference, answer sentence selection, and sentence classification, outperforming state-of-the-art recurrent and recursive neural networks .</p>","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"1 ","pages":"11-21"},"PeriodicalIF":0.0,"publicationDate":"2017-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5657441/pdf/nihms912010.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35552522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Proceedings of the conference. Association for Computational Linguistics. Meeting
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1