Pub Date : 2021-09-30DOI: 10.1017/9781108924184.015
Bang Liu, Lingfei Wu
Natural language processing (NLP) and understanding aim to read from unformatted text to accomplish different tasks. While word embeddings learned by deep neural networks are widely used, the underlying linguistic and semantic structures of text pieces cannot be fully exploited in these representations. Graph is a natural way to capture the connections between different text pieces, such as entities, sentences, and documents. To overcome the limits in vector space models, researchers combine deep learning models with graph-structured representations for various tasks in NLP and text mining. Such combinations help to make full use of both the structural information in text and the representation learning ability of deep neural networks. In this chapter, we introduce the various graph representations that are extensively used in NLP, and show how different NLP tasks can be tackled from a graph perspective. We summarize recent research works on graph-based NLP, and discuss two case studies related to graph-based text clustering, matching, and multihop machine reading comprehension in detail. Finally, we provide a synthesis about the important open problems of this subfield.
{"title":"Graph Neural Networks in Natural Language Processing","authors":"Bang Liu, Lingfei Wu","doi":"10.1017/9781108924184.015","DOIUrl":"https://doi.org/10.1017/9781108924184.015","url":null,"abstract":"Natural language processing (NLP) and understanding aim to read from unformatted text to accomplish different tasks. While word embeddings learned by deep neural networks are widely used, the underlying linguistic and semantic structures of text pieces cannot be fully exploited in these representations. Graph is a natural way to capture the connections between different text pieces, such as entities, sentences, and documents. To overcome the limits in vector space models, researchers combine deep learning models with graph-structured representations for various tasks in NLP and text mining. Such combinations help to make full use of both the structural information in text and the representation learning ability of deep neural networks. In this chapter, we introduce the various graph representations that are extensively used in NLP, and show how different NLP tasks can be tackled from a graph perspective. We summarize recent research works on graph-based NLP, and discuss two case studies related to graph-based text clustering, matching, and multihop machine reading comprehension in detail. Finally, we provide a synthesis about the important open problems of this subfield.","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115990554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-30DOI: 10.1017/9781108924184.009
Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, Le Song
Deep Learning has become one of the most dominant approaches in Artificial Intelligence research today. Although conventional deep learning techniques have achieved huge successes on Euclidean data such as images, or sequence data such as text, there are many applications that are naturally or best represented with a graph structure. This gap has driven a tide in research for deep learning on graphs, among them Graph Neural Networks (GNNs) are the most successful in coping with various learning tasks across a large number of application domains. In this chapter, we will systematically organize the existing research of GNNs along three axes: foundations, frontiers, and applications. We will introduce the fundamental aspects of GNNs ranging from the popular models and their expressive powers, to the scalability, interpretability and robustness of GNNs. Then, we will discuss various frontier research, ranging from graph classification and link prediction, to graph generation and transformation, graph matching and graph structure learning. Based on them, we further summarize the basic procedures which exploit full use of various GNNs for a large number of applications. Finally, we provide the organization of our book and summarize the roadmap of the various research topics of GNNs. Lingfei Wu JD.COM Silicon Valley Research Center, e-mail: lwu@email.wm.edu Peng Cui Department of Computer Science, Tsinghua University, e-mail: cuip@tsinghua.edu.cn Jian Pei Department of Computer Science, Simon Fraser University, e-mail: jpei@cs.sfu.ca Liang Zhao Department of Computer Science, Emory University, e-mail: liang.zhao@emory.edu Le Song Mohamed bin Zayed University of Artificial Intelligence, e-mail: dasongle@gmail.com
{"title":"Graph Neural Networks","authors":"Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, Le Song","doi":"10.1017/9781108924184.009","DOIUrl":"https://doi.org/10.1017/9781108924184.009","url":null,"abstract":"Deep Learning has become one of the most dominant approaches in Artificial Intelligence research today. Although conventional deep learning techniques have achieved huge successes on Euclidean data such as images, or sequence data such as text, there are many applications that are naturally or best represented with a graph structure. This gap has driven a tide in research for deep learning on graphs, among them Graph Neural Networks (GNNs) are the most successful in coping with various learning tasks across a large number of application domains. In this chapter, we will systematically organize the existing research of GNNs along three axes: foundations, frontiers, and applications. We will introduce the fundamental aspects of GNNs ranging from the popular models and their expressive powers, to the scalability, interpretability and robustness of GNNs. Then, we will discuss various frontier research, ranging from graph classification and link prediction, to graph generation and transformation, graph matching and graph structure learning. Based on them, we further summarize the basic procedures which exploit full use of various GNNs for a large number of applications. Finally, we provide the organization of our book and summarize the roadmap of the various research topics of GNNs. Lingfei Wu JD.COM Silicon Valley Research Center, e-mail: lwu@email.wm.edu Peng Cui Department of Computer Science, Tsinghua University, e-mail: cuip@tsinghua.edu.cn Jian Pei Department of Computer Science, Simon Fraser University, e-mail: jpei@cs.sfu.ca Liang Zhao Department of Computer Science, Emory University, e-mail: liang.zhao@emory.edu Le Song Mohamed bin Zayed University of Artificial Intelligence, e-mail: dasongle@gmail.com","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125995708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-30DOI: 10.1017/9781108924184.003
{"title":"Deep Learning on Graphs: An Introduction","authors":"","doi":"10.1017/9781108924184.003","DOIUrl":"https://doi.org/10.1017/9781108924184.003","url":null,"abstract":"","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128322969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-30DOI: 10.1017/9781108924184.012
In the earlier chapters, we have discussed graph neural network models focusing on simple graphs where the graphs are static and have only one type of nodes and one type of edges. However, graphs in many real-world applications are much more complicated. They typically have multiple types of nodes, edges, unique structures, and often are dynamic. As a consequence, these complex graphs present more intricate patterns that are beyond the capacity of the aforementioned graph neural network models on simple graphs. Thus, dedicated efforts are desired to design graph neural network models for complex graphs. These efforts can significantly impact the successful adoption and use of GNNs in a broader range of applications. In this chapter, using complex graphs introduced in Section 2.6 as examples, we discuss the methods to extend the graph neural network models to capture more sophisticated patterns. More specifically, we describe more advanced graph filters designed for complex graphs to capture their specific properties.
{"title":"Graph Neural Networks for Complex Graphs","authors":"","doi":"10.1017/9781108924184.012","DOIUrl":"https://doi.org/10.1017/9781108924184.012","url":null,"abstract":"In the earlier chapters, we have discussed graph neural network models focusing on simple graphs where the graphs are static and have only one type of nodes and one type of edges. However, graphs in many real-world applications are much more complicated. They typically have multiple types of nodes, edges, unique structures, and often are dynamic. As a consequence, these complex graphs present more intricate patterns that are beyond the capacity of the aforementioned graph neural network models on simple graphs. Thus, dedicated efforts are desired to design graph neural network models for complex graphs. These efforts can significantly impact the successful adoption and use of GNNs in a broader range of applications. In this chapter, using complex graphs introduced in Section 2.6 as examples, we discuss the methods to extend the graph neural network models to capture more sophisticated patterns. More specifically, we describe more advanced graph filters designed for complex graphs to capture their specific properties.","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128295450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-30DOI: 10.1017/9781108924184.013
{"title":"Beyond GNNs: More Deep Models on Graphs","authors":"","doi":"10.1017/9781108924184.013","DOIUrl":"https://doi.org/10.1017/9781108924184.013","url":null,"abstract":"","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121068760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-30DOI: 10.1017/9781108924184.010
As the generalizations of traditional DNNs to graphs, GNNs inherit both advantages and disadvantages of traditional DNNs. Like traditional DNNs, GNNs have been shown to be effective in many graph-related tasks such as nodefocused and graph-focused tasks. Traditional DNNs have been demonstrated to be vulnerable to dedicated designed adversarial attacks (Goodfellow et al., 2014b; Xu et al., 2019b). Under adversarial attacks, the victimized samples are perturbed in such a way that they are not easily noticeable, but they can lead to wrong results. It is increasingly evident that GNNs also inherit this drawback. The adversary can generate graph adversarial perturbations by manipulating the graph structure or node features to fool the GNN models. This limitation of GNNs has arisen immense concerns on adopting them in safety-critical applications such as financial systems and risk management. For example, in a credit scoring system, fraudsters can fake connections with several high-credit customers to evade the fraudster detection models; and spammers can easily create fake followers to increase the chance of fake news being recommended and spread. Therefore, we have witnessed more and more research attention to graph adversarial attacks and their countermeasures. In this chapter, we first introduce concepts and definitions of graph adversarial attacks and detail some representative adversarial attack methods on graphs. Then, we discuss representative defense techniques against these adversarial attacks.
作为传统深度神经网络对图的推广，gnn继承了传统深度神经网络的优点和缺点。与传统的深度神经网络一样，gnn已被证明在许多与图相关的任务中是有效的，例如节点分散和以图为中心的任务。传统的深度神经网络已被证明容易受到专门设计的对抗性攻击(Goodfellow等人，2014;Xu et al.， 2019b)。在对抗性攻击下，受害样本受到干扰，不容易被注意到，但它们可能导致错误的结果。越来越明显的是，gnn也继承了这个缺点。攻击者可以通过操纵图结构或节点特征来产生图对抗性扰动来欺骗GNN模型。gnn的这种局限性引起了人们对在金融系统和风险管理等安全关键应用中采用它们的极大关注。例如，在信用评分系统中，欺诈者可以伪造与多个高信用客户的联系，以逃避欺诈者检测模型;垃圾邮件发送者可以很容易地创建假粉丝，以增加假新闻被推荐和传播的机会。因此，对图对抗攻击及其对策的研究越来越受到重视。在本章中，我们首先介绍了图对抗攻击的概念和定义，并详细介绍了一些典型的图对抗攻击方法。然后，我们讨论了针对这些对抗性攻击的代表性防御技术。
{"title":"Robust Graph Neural Networks","authors":"","doi":"10.1017/9781108924184.010","DOIUrl":"https://doi.org/10.1017/9781108924184.010","url":null,"abstract":"As the generalizations of traditional DNNs to graphs, GNNs inherit both advantages and disadvantages of traditional DNNs. Like traditional DNNs, GNNs have been shown to be effective in many graph-related tasks such as nodefocused and graph-focused tasks. Traditional DNNs have been demonstrated to be vulnerable to dedicated designed adversarial attacks (Goodfellow et al., 2014b; Xu et al., 2019b). Under adversarial attacks, the victimized samples are perturbed in such a way that they are not easily noticeable, but they can lead to wrong results. It is increasingly evident that GNNs also inherit this drawback. The adversary can generate graph adversarial perturbations by manipulating the graph structure or node features to fool the GNN models. This limitation of GNNs has arisen immense concerns on adopting them in safety-critical applications such as financial systems and risk management. For example, in a credit scoring system, fraudsters can fake connections with several high-credit customers to evade the fraudster detection models; and spammers can easily create fake followers to increase the chance of fake news being recommended and spread. Therefore, we have witnessed more and more research attention to graph adversarial attacks and their countermeasures. In this chapter, we first introduce concepts and definitions of graph adversarial attacks and detail some representative adversarial attack methods on graphs. Then, we discuss representative defense techniques against these adversarial attacks.","PeriodicalId":254746,"journal":{"name":"Deep Learning on Graphs","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114967646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：