LGAT: a light graph attention network focusing on message passing for semi-supervised node classification

IF 3.3 3区计算机科学 Q2 COMPUTER SCIENCE, THEORY & METHODS Computing Pub Date : 2024-02-16 DOI:10.1007/s00607-024-01261-6

{"title":"LGAT: a light graph attention network focusing on message passing for semi-supervised node classification","authors":"","doi":"10.1007/s00607-024-01261-6","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Deep learning has shown superior performance in various applications. The emergence of graph convolution neural networks (GCNs) enables deep learning to learn the latent representation from graph-structured data with rich attributes. To be specific, the message passing mechanism of GCNs can aggregate and update messages through the topological relationship between nodes in a graph. The graph attention network (GAT) introduces the attention mechanism into GCNs when aggregating messages and achieves significant performance on the node classification task. However, focusing on each node in the neighborhood, GAT becomes extremely complex. In addition, although stacking network layers could obtain a wider receptive field, it also brings high time cost and leads to the difficulty of training. To handle this problem, this paper only divides the messages into two types, i.e. self message and neighborhood message. The neighborhood message comes from the neighborhood with(out) self-loop while the self message comes from the node itself. Then, we design a light attention mechanism that only focuses on two weights, one for the self message, and the other for the neighborhood message, to adaptively reveal the different contributions of messages from a node as well as its neighborhood. In addition, we also adopt linear propagation, a shallow and efficient method, to aggregate messages from distant neighbors and thus obtain a wider neighborhood receiving field. To verify the effectiveness of our proposed approach, extensive experiments have been conducted on the semi-supervised node classification task. Results show that our proposed approach achieves comparable or even better performance than the baseline methods with complicated GCN structures on the benchmark datasets. Specifically, the proposed light attention mechanism focusing on message passing exhibits a great efficiency improvement with the training time cost less than half of GAT.</p>","PeriodicalId":10718,"journal":{"name":"Computing","volume":"10 1","pages":""},"PeriodicalIF":3.3000,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computing","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s00607-024-01261-6","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}

引用次数: 0

Abstract

Deep learning has shown superior performance in various applications. The emergence of graph convolution neural networks (GCNs) enables deep learning to learn the latent representation from graph-structured data with rich attributes. To be specific, the message passing mechanism of GCNs can aggregate and update messages through the topological relationship between nodes in a graph. The graph attention network (GAT) introduces the attention mechanism into GCNs when aggregating messages and achieves significant performance on the node classification task. However, focusing on each node in the neighborhood, GAT becomes extremely complex. In addition, although stacking network layers could obtain a wider receptive field, it also brings high time cost and leads to the difficulty of training. To handle this problem, this paper only divides the messages into two types, i.e. self message and neighborhood message. The neighborhood message comes from the neighborhood with(out) self-loop while the self message comes from the node itself. Then, we design a light attention mechanism that only focuses on two weights, one for the self message, and the other for the neighborhood message, to adaptively reveal the different contributions of messages from a node as well as its neighborhood. In addition, we also adopt linear propagation, a shallow and efficient method, to aggregate messages from distant neighbors and thus obtain a wider neighborhood receiving field. To verify the effectiveness of our proposed approach, extensive experiments have been conducted on the semi-supervised node classification task. Results show that our proposed approach achieves comparable or even better performance than the baseline methods with complicated GCN structures on the benchmark datasets. Specifically, the proposed light attention mechanism focusing on message passing exhibits a great efficiency improvement with the training time cost less than half of GAT.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

LGAT：侧重于信息传递的轻型图注意网络，用于半监督节点分类

摘要深度学习在各种应用中表现出卓越的性能。图卷积神经网络（GCN）的出现使深度学习能够从具有丰富属性的图结构数据中学习潜在表示。具体来说，GCN 的消息传递机制可以通过图中节点之间的拓扑关系来聚合和更新消息。图注意力网络（GAT）在聚合信息时将注意力机制引入了 GCN，并在节点分类任务中取得了显著的性能。然而，由于关注邻域中的每个节点，GAT 变得异常复杂。此外，虽然堆叠网络层可以获得更宽的感受野，但也带来了高昂的时间成本和训练难度。为了解决这个问题，本文只将信息分为两类，即自身信息和邻域信息。邻域信息来自有（无）自循环的邻域，而自身信息则来自节点本身。然后，我们设计了一种轻关注机制，它只关注两个权重，一个是自身信息的权重，另一个是邻域信息的权重，从而自适应地揭示来自节点及其邻域的信息的不同贡献。此外，我们还采用了线性传播这种浅显而高效的方法来聚合来自远邻的信息，从而获得更广阔的邻域接收域。为了验证我们提出的方法的有效性，我们在半监督节点分类任务中进行了大量实验。结果表明，在基准数据集上，我们提出的方法取得了与具有复杂 GCN 结构的基准方法相当甚至更好的性能。具体来说，我们提出的轻关注机制以消息传递为重点，大大提高了效率，其训练时间成本不到 GAT 的一半。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Computing 工程技术-计算机：理论方法

CiteScore

8.20

自引率

2.70%

发文量

107

审稿时长

3 months

期刊介绍： Computing publishes original papers, short communications and surveys on all fields of computing. The contributions should be written in English and may be of theoretical or applied nature, the essential criteria are computational relevance and systematic foundation of results.