Pub Date : 2024-10-24DOI: 10.1016/j.neunet.2024.106829
Dan Su , Long Jin , Jun Wang
Sharpness-aware minimization (SAM) aims to enhance model generalization by minimizing the sharpness of the loss function landscape, leading to a robust model performance. To protect sensitive information and enhance privacy, prevailing approaches add noise to models. However, additive noises would inevitably degrade the generalization and robustness of the model. In this paper, we propose a noise-resistant SAM method, based on a noise-resistant parameter update rule. We analyze the convergence and noise resistance properties of the proposed method under noisy conditions. We elaborate on experimental results with several networks on various benchmark datasets to demonstrate the advantages of the proposed method with respect to model generalization and privacy protection.
锐度感知最小化（SAM）旨在通过最小化损失函数景观的锐度来增强模型的泛化，从而获得稳健的模型性能。为了保护敏感信息和提高私密性，普遍采用的方法是在模型中添加噪声。然而，添加噪声不可避免地会降低模型的泛化和鲁棒性。本文基于抗噪参数更新规则，提出了一种抗噪 SAM 方法。我们分析了所提方法在噪声条件下的收敛性和抗噪声特性。我们详细阐述了几个网络在各种基准数据集上的实验结果，以证明所提方法在模型泛化和隐私保护方面的优势。
{"title":"Noise-resistant sharpness-aware minimization in deep learning","authors":"Dan Su , Long Jin , Jun Wang","doi":"10.1016/j.neunet.2024.106829","DOIUrl":"10.1016/j.neunet.2024.106829","url":null,"abstract":"<div><div>Sharpness-aware minimization (SAM) aims to enhance model generalization by minimizing the sharpness of the loss function landscape, leading to a robust model performance. To protect sensitive information and enhance privacy, prevailing approaches add noise to models. However, additive noises would inevitably degrade the generalization and robustness of the model. In this paper, we propose a noise-resistant SAM method, based on a noise-resistant parameter update rule. We analyze the convergence and noise resistance properties of the proposed method under noisy conditions. We elaborate on experimental results with several networks on various benchmark datasets to demonstrate the advantages of the proposed method with respect to model generalization and privacy protection.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106829"},"PeriodicalIF":6.0,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142565325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multivariate time series exhibit complex patterns and structures involving interactions among multiple variables and long-term temporal dependencies, making multivariate long sequence time series forecasting (MLSTF) exceptionally challenging. Despite significant progress in Transformer-based methods in the MLSTF domain, many models still rely on stacked encoder–decoder architectures to capture complex time series patterns. This leads to increased computational complexity and overlooks spatial pattern information in multivariate time series, thereby limiting the model’s performance. To address these challenges, we propose RFNet, a lightweight model based on recurrent representation and feature enhancement. We partition the time series into fixed-size subsequences to retain local contextual temporal pattern information and cross-variable spatial pattern information. The recurrent representation module employs gate attention mechanisms and memory units to capture local information of the subsequences and obtain long-term correlation information of the input sequence by integrating information from different memory units. Meanwhile, we utilize a shared multi-layer perceptron (MLP) to capture global pattern information of the input sequence. The feature enhancement module explicitly extracts complex spatial patterns in the time series by transforming the input sequence. We validate the performance of RFNet on ten real-world datasets. The results demonstrate an improvement of approximately 55.3% over state-of-the-art MLSTF models, highlighting its significant advantage in addressing multivariate long sequence time series forecasting problems.
{"title":"RFNet: Multivariate long sequence time-series forecasting based on recurrent representation and feature enhancement","authors":"Dandan Zhang , Zhiqiang Zhang , Nanguang Chen , Yun Wang","doi":"10.1016/j.neunet.2024.106800","DOIUrl":"10.1016/j.neunet.2024.106800","url":null,"abstract":"<div><div>Multivariate time series exhibit complex patterns and structures involving interactions among multiple variables and long-term temporal dependencies, making multivariate long sequence time series forecasting (MLSTF) exceptionally challenging. Despite significant progress in Transformer-based methods in the MLSTF domain, many models still rely on stacked encoder–decoder architectures to capture complex time series patterns. This leads to increased computational complexity and overlooks spatial pattern information in multivariate time series, thereby limiting the model’s performance. To address these challenges, we propose RFNet, a lightweight model based on recurrent representation and feature enhancement. We partition the time series into fixed-size subsequences to retain local contextual temporal pattern information and cross-variable spatial pattern information. The recurrent representation module employs gate attention mechanisms and memory units to capture local information of the subsequences and obtain long-term correlation information of the input sequence by integrating information from different memory units. Meanwhile, we utilize a shared multi-layer perceptron (MLP) to capture global pattern information of the input sequence. The feature enhancement module explicitly extracts complex spatial patterns in the time series by transforming the input sequence. We validate the performance of RFNet on ten real-world datasets. The results demonstrate an improvement of approximately 55.3% over state-of-the-art MLSTF models, highlighting its significant advantage in addressing multivariate long sequence time series forecasting problems.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106800"},"PeriodicalIF":6.0,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142563530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-23DOI: 10.1016/j.neunet.2024.106811
Linlu Dong, Jun Wang
Infrared and visible light image fusion can solve the limitations of single-type visual sensors and can boost the target detection performance. However, since the traditional fusion strategy lacks the controllability and feedback mechanism, the fusion model cannot precisely perceive the relationship between the requirements of the fusion task, the fused image quality, and the source image features. To this end, this paper establishes a fusion model based on the optimal controlled object and control mode called FusionOC. This method establishes two types of mathematical models of the controlled objects by verifying the factors and conflicts affecting the quality of the fused image. It combines the image fusion model with the quality evaluation function to determine the two control factors separately. At the same time, two proportional-integral-derivative (PID) control and regulation modes based on the backpropagation (BP) neural network are designed according to the control factor characteristics. The fusion system can adaptively select the regulation mode to regulate the control factor according to the user requirements or the task to make the fusion system perceive the connection between the fusion task and the result. Besides, the fusion model employs the feedback mechanism of the control system to perceive the feature difference between the fusion result and the source image, realize the guidance of the source image feature to the entire fusion process, and improve the fusion algorithm's generalization ability and intelligence level when handling different fusion tasks. Experimental results on multiple public datasets demonstrate the advantages of FusionOC over advanced methods. Meanwhile, the benefits of our fusion results in object detection tasks have been demonstrated.
{"title":"FusionOC: Research on optimal control method for infrared and visible light image fusion","authors":"Linlu Dong, Jun Wang","doi":"10.1016/j.neunet.2024.106811","DOIUrl":"10.1016/j.neunet.2024.106811","url":null,"abstract":"<div><div>Infrared and visible light image fusion can solve the limitations of single-type visual sensors and can boost the target detection performance. However, since the traditional fusion strategy lacks the controllability and feedback mechanism, the fusion model cannot precisely perceive the relationship between the requirements of the fusion task, the fused image quality, and the source image features. To this end, this paper establishes a fusion model based on the optimal controlled object and control mode called FusionOC. This method establishes two types of mathematical models of the controlled objects by verifying the factors and conflicts affecting the quality of the fused image. It combines the image fusion model with the quality evaluation function to determine the two control factors separately. At the same time, two proportional-integral-derivative (PID) control and regulation modes based on the backpropagation (BP) neural network are designed according to the control factor characteristics. The fusion system can adaptively select the regulation mode to regulate the control factor according to the user requirements or the task to make the fusion system perceive the connection between the fusion task and the result. Besides, the fusion model employs the feedback mechanism of the control system to perceive the feature difference between the fusion result and the source image, realize the guidance of the source image feature to the entire fusion process, and improve the fusion algorithm's generalization ability and intelligence level when handling different fusion tasks. Experimental results on multiple public datasets demonstrate the advantages of FusionOC over advanced methods. Meanwhile, the benefits of our fusion results in object detection tasks have been demonstrated.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106811"},"PeriodicalIF":6.0,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142561306","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-23DOI: 10.1016/j.neunet.2024.106779
Kaiyi Xu , Minhui Wang , Xin Zou , Jingjing Liu , Ao Wei , Jiajia Chen , Chang Tang
Identifying the frequencies of drug-side effects is crucial for assessing drug risk-benefit. However, accurately determining these frequencies remains challenging due to the limitations of time and scale in clinical randomized controlled trials. As a result, several computational methods have been proposed to address these issues. Nonetheless, two primary problems still persist. Firstly, most of these methods face challenges in generating accurate predictions for novel drugs, as they heavily depend on the interaction graph between drugs and side effects (SEs) within their modeling framework. Secondly, some previous methods often simply concatenate the features of drugs and SEs, which fails to effectively capture their underlying association. In this work, we present HSTrans, a novel approach that treats drugs and SEs as sets of substructures, leveraging a transformer encoder for unified substructure embedding and incorporating an interaction module for association capture. Specifically, HSTrans extracts drug substructures through a specialized algorithm and identifies effective substructures for each SE by employing an indicator that measures the importance of each substructure and SE. Additionally, HSTrans applies convolutional neural network (CNN) in the interaction module to capture complex relationships between drugs and SEs. Experimental results on datasets from Galeano et al.’s study demonstrate that the proposed method outperforms other state-of-the-art approaches. The demo codes for HSTrans are available at https://github.com/Dtdtxuky/HSTrans/tree/master.
确定药物副作用的频率对于评估药物的风险-效益至关重要。然而，由于临床随机对照试验在时间和规模上的限制，准确确定这些频率仍具有挑战性。因此，人们提出了几种计算方法来解决这些问题。然而，两个主要问题仍然存在。首先，大多数这些方法在对新药进行准确预测时都面临挑战，因为它们在建模框架内严重依赖于药物与副作用（SEs）之间的相互作用图。其次，以前的一些方法往往只是简单地将药物和副作用的特征串联起来，无法有效捕捉它们之间的内在联系。在这项工作中，我们提出了 HSTrans，这是一种将药物和副作用作为子结构集来处理的新方法，它利用变换器编码器进行统一的子结构嵌入，并结合了一个用于关联捕捉的交互模块。具体来说，HSTrans 通过专门的算法提取药物子结构，并通过采用衡量每个子结构和 SE 重要性的指标来识别每个 SE 的有效子结构。此外，HSTrans 还在交互模块中应用了卷积神经网络 (CNN)，以捕捉药物与 SE 之间的复杂关系。在 Galeano 等人的研究数据集上的实验结果表明，所提出的方法优于其他最先进的方法。HSTrans 的演示代码请访问 https://github.com/Dtdtxuky/HSTrans/tree/master。
{"title":"HSTrans: Homogeneous substructures transformer for predicting frequencies of drug-side effects","authors":"Kaiyi Xu , Minhui Wang , Xin Zou , Jingjing Liu , Ao Wei , Jiajia Chen , Chang Tang","doi":"10.1016/j.neunet.2024.106779","DOIUrl":"10.1016/j.neunet.2024.106779","url":null,"abstract":"<div><div>Identifying the frequencies of drug-side effects is crucial for assessing drug risk-benefit. However, accurately determining these frequencies remains challenging due to the limitations of time and scale in clinical randomized controlled trials. As a result, several computational methods have been proposed to address these issues. Nonetheless, two primary problems still persist. Firstly, most of these methods face challenges in generating accurate predictions for novel drugs, as they heavily depend on the interaction graph between drugs and side effects (SEs) within their modeling framework. Secondly, some previous methods often simply concatenate the features of drugs and SEs, which fails to effectively capture their underlying association. In this work, we present HSTrans, a novel approach that treats drugs and SEs as sets of substructures, leveraging a transformer encoder for unified substructure embedding and incorporating an interaction module for association capture. Specifically, HSTrans extracts drug substructures through a specialized algorithm and identifies effective substructures for each SE by employing an indicator that measures the importance of each substructure and SE. Additionally, HSTrans applies convolutional neural network (CNN) in the interaction module to capture complex relationships between drugs and SEs. Experimental results on datasets from Galeano et al.’s study demonstrate that the proposed method outperforms other state-of-the-art approaches. The demo codes for HSTrans are available at <span><span>https://github.com/Dtdtxuky/HSTrans/tree/master</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106779"},"PeriodicalIF":6.0,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142565352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-22DOI: 10.1016/j.neunet.2024.106824
Cong Zheng , Yixuan Song
With the rapid explosion of online news and user population, personalized news recommender systems have proved to be efficient ways of alleviating information overload problems by suggesting information which attracts users in line with their tastes. Exploring relationships among words and news is critical to structurally model users’ latent tastes including interested domains, while selecting informative words and news can directly reflect users’ interests. Most of the current studies do not provide an effective framework that combines distilling users’ interested latent spaces and explicit points systematically. Moreover, introducing more advanced techniques to merely chase accuracy has become a universal phenomenon. In this study, we design a Personalized Multi-Head Self-Attention Network (PMSN) for news recommendation, which combines multi-head self-attention network with personalized attention mechanism from both word and news levels. Multi-head self-attention mechanism is used to model interactions among words and news, exploring latent interests. Personalized attention mechanism is applied by embedding users’ IDs to highlight informative words and news, which can enhance the interpretability of personalization. Comprehensive experiments conducted using two real-world datasets demonstrate that PMSN efficiently outperforms state-of-the-art methods in terms of recommendation accuracy, without complicated structure design and exhausted even external resources consumption. Furthermore, visualized case study validates that attention mechanism indeed increases the interpretability.
随着网络新闻和用户数量的快速增长，个性化新闻推荐系统已被证明是缓解信息过载问题的有效方法，它可以推荐符合用户口味的信息来吸引用户。探索词语和新闻之间的关系对于结构化地模拟用户的潜在品味（包括感兴趣的领域）至关重要，而选择信息丰富的词语和新闻则能直接反映用户的兴趣。目前的大多数研究都没有提供一个有效的框架，将用户感兴趣的潜在空间和显性点系统地结合起来。此外，引入更先进的技术来单纯追求准确率已成为普遍现象。在本研究中，我们设计了一种用于新闻推荐的个性化多头自我关注网络（PMSN），该网络将多头自我关注网络与个性化关注机制相结合，从单词和新闻两个层面进行推荐。多头自我关注机制用于建立词语和新闻之间的互动模型，探索潜在兴趣。个性化关注机制通过嵌入用户 ID 来突出显示有信息量的词语和新闻，从而增强个性化的可解释性。利用两个真实数据集进行的综合实验表明，PMSN 在推荐准确性方面有效地超越了最先进的方法，而且不需要复杂的结构设计，甚至不需要消耗外部资源。此外，可视化案例研究也验证了关注机制确实提高了可解释性。
{"title":"Personalized multi-head self-attention network for news recommendation","authors":"Cong Zheng , Yixuan Song","doi":"10.1016/j.neunet.2024.106824","DOIUrl":"10.1016/j.neunet.2024.106824","url":null,"abstract":"<div><div>With the rapid explosion of online news and user population, personalized news recommender systems have proved to be efficient ways of alleviating information overload problems by suggesting information which attracts users in line with their tastes. Exploring relationships among words and news is critical to structurally model users’ latent tastes including interested domains, while selecting informative words and news can directly reflect users’ interests. Most of the current studies do not provide an effective framework that combines distilling users’ interested latent spaces and explicit points systematically. Moreover, introducing more advanced techniques to merely chase accuracy has become a universal phenomenon. In this study, we design a <strong>P</strong>ersonalized <strong>M</strong>ulti-Head <strong>S</strong>elf-Attention <strong>N</strong>etwork (<strong>PMSN</strong>) for news recommendation, which combines multi-head self-attention network with personalized attention mechanism from both word and news levels. Multi-head self-attention mechanism is used to model interactions among words and news, exploring latent interests. Personalized attention mechanism is applied by embedding users’ IDs to highlight informative words and news, which can enhance the interpretability of personalization. Comprehensive experiments conducted using two real-world datasets demonstrate that PMSN efficiently outperforms state-of-the-art methods in terms of recommendation accuracy, without complicated structure design and exhausted even external resources consumption. Furthermore, visualized case study validates that attention mechanism indeed increases the interpretability.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106824"},"PeriodicalIF":6.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-22DOI: 10.1016/j.neunet.2024.106826
Elsa Cardoso-Bihlo, Alex Bihlo
We introduce a method for training exactly conservative physics-informed neural networks and physics-informed deep operator networks for dynamical systems, that is, for ordinary differential equations. The method employs a projection-based technique that maps a candidate solution learned by the neural network solver for any given dynamical system possessing at least one first integral onto an invariant manifold. We illustrate that exactly conservative physics-informed neural network solvers and physics-informed deep operator networks for dynamical systems vastly outperform their non-conservative counterparts for several real-world problems from the mathematical sciences.
{"title":"Exactly conservative physics-informed neural networks and deep operator networks for dynamical systems","authors":"Elsa Cardoso-Bihlo, Alex Bihlo","doi":"10.1016/j.neunet.2024.106826","DOIUrl":"10.1016/j.neunet.2024.106826","url":null,"abstract":"<div><div>We introduce a method for training exactly conservative physics-informed neural networks and physics-informed deep operator networks for dynamical systems, that is, for ordinary differential equations. The method employs a projection-based technique that maps a candidate solution learned by the neural network solver for any given dynamical system possessing at least one first integral onto an invariant manifold. We illustrate that exactly conservative physics-informed neural network solvers and physics-informed deep operator networks for dynamical systems vastly outperform their non-conservative counterparts for several real-world problems from the mathematical sciences.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106826"},"PeriodicalIF":6.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142607189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-22DOI: 10.1016/j.neunet.2024.106827
Kai Zhou , Jinglong Fang , Dan Wei , Wen Wu , Rui Hu
Sparsely annotated image segmentation has attracted increasing attention due to its low labeling cost. However, existing weakly-supervised shadow detection methods require complex training procedures, and there is still a significant performance gap compared to fully-supervised methods. This paper summarizes two current challenges in sparsely annotated shadow detection, i.e., weak supervision diffusion and poor structure recovery, and attempts to alleviate them. To this end, we propose a one-stage weakly-supervised learning framework to facilitate sparsely annotated shadow detection. Specifically, we first design a simple yet effective semantic affinity module (SAM) that adaptively propagates scribble supervision to unlabeled regions using a gradient diffusion scheme. Then, to better recover shadow structures, we introduce a feature-guided edge-aware loss, which leverages higher-level semantic relations to perceive shadow boundaries, while avoiding interference from ambiguous regions. Finally, we present an intensity-guided structure consistency loss to ensure that the same images with different brightness are predicted to be consistent shadow masks, which can be regarded as a self-consistent mechanism to improve the model’s generalization ability. Experimental results on three benchmark datasets demonstrate that our approach significantly outperforms previous weakly-supervised methods and achieves competitive performance in comparison to recent state-of-the-art fully-supervised methods.
{"title":"Exploring better sparsely annotated shadow detection","authors":"Kai Zhou , Jinglong Fang , Dan Wei , Wen Wu , Rui Hu","doi":"10.1016/j.neunet.2024.106827","DOIUrl":"10.1016/j.neunet.2024.106827","url":null,"abstract":"<div><div>Sparsely annotated image segmentation has attracted increasing attention due to its low labeling cost. However, existing weakly-supervised shadow detection methods require complex training procedures, and there is still a significant performance gap compared to fully-supervised methods. This paper summarizes two current challenges in sparsely annotated shadow detection, i.e., weak supervision diffusion and poor structure recovery, and attempts to alleviate them. To this end, we propose a one-stage weakly-supervised learning framework to facilitate sparsely annotated shadow detection. Specifically, we first design a simple yet effective semantic affinity module (SAM) that adaptively propagates scribble supervision to unlabeled regions using a gradient diffusion scheme. Then, to better recover shadow structures, we introduce a feature-guided edge-aware loss, which leverages higher-level semantic relations to perceive shadow boundaries, while avoiding interference from ambiguous regions. Finally, we present an intensity-guided structure consistency loss to ensure that the same images with different brightness are predicted to be consistent shadow masks, which can be regarded as a self-consistent mechanism to improve the model’s generalization ability. Experimental results on three benchmark datasets demonstrate that our approach significantly outperforms previous weakly-supervised methods and achieves competitive performance in comparison to recent state-of-the-art fully-supervised methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106827"},"PeriodicalIF":6.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-22DOI: 10.1016/j.neunet.2024.106828
Li Meng , Yunfei He , Chenyuan Sun , Lishan Huang , Taizhang Hu , Fei Yang
Multi-drug combination therapies are increasingly used for complex diseases but carry risks of harmful drug interactions. Effective drug–drug interaction prediction (DDIP) is essential for assessing risks among numerous drug pairs. Most DDIP methods involve two main steps: drug representation and drug pair interaction extraction, respectively challenged by the loss of personalized drug information and the need for differentiated interaction data, and are rarely studied. Specifically, personalized drug information refers to the distinct features of each drug. These properties can be easily confused by neighboring information during graph propagation. This issue is especially prominent in drug interaction graphs with long-tail distributions, which poses challenges for personalized drug information learning. Furthermore, it is crucial to learn interactions with differentiation in order to identify diverse drug relationships. Some methods simply concatenate drug features, often ignoring the differences of different drug relationships, while other methods based on substructures rely on professional pharmacological knowledge and are computationally complex. To address these issues, we propose a novel method, learning personalized Drug Features and differentiated Drug-Pair interaction information for drug–drug interaction prediction (DFPDDI). This approach employs a contrastive learning network with edge-aware augmentations and mutual information estimators to capture personalized drug features across various graph distributions. Furthermore, it applies a mutual information constraint to drug-pair representations, enhancing the accuracy of interaction predictions by better distinguishing between different types of drug relationships. The results evaluated on three public datasets demonstrate competitive performance compared to baselines. It also shows potential for accurate predictions, particularly in imbalanced-distribution graphs.
{"title":"Learning personalized drug features and differentiated drug-pair interaction information for drug–drug interaction prediction","authors":"Li Meng , Yunfei He , Chenyuan Sun , Lishan Huang , Taizhang Hu , Fei Yang","doi":"10.1016/j.neunet.2024.106828","DOIUrl":"10.1016/j.neunet.2024.106828","url":null,"abstract":"<div><div>Multi-drug combination therapies are increasingly used for complex diseases but carry risks of harmful drug interactions. Effective drug–drug interaction prediction (DDIP) is essential for assessing risks among numerous drug pairs. Most DDIP methods involve two main steps: drug representation and drug pair interaction extraction, respectively challenged by the loss of personalized drug information and the need for differentiated interaction data, and are rarely studied. Specifically, personalized drug information refers to the distinct features of each drug. These properties can be easily confused by neighboring information during graph propagation. This issue is especially prominent in drug interaction graphs with long-tail distributions, which poses challenges for personalized drug information learning. Furthermore, it is crucial to learn interactions with differentiation in order to identify diverse drug relationships. Some methods simply concatenate drug features, often ignoring the differences of different drug relationships, while other methods based on substructures rely on professional pharmacological knowledge and are computationally complex. To address these issues, we propose a novel method, learning personalized Drug Features and differentiated Drug-Pair interaction information for drug–drug interaction prediction (DFPDDI). This approach employs a contrastive learning network with edge-aware augmentations and mutual information estimators to capture personalized drug features across various graph distributions. Furthermore, it applies a mutual information constraint to drug-pair representations, enhancing the accuracy of interaction predictions by better distinguishing between different types of drug relationships. The results evaluated on three public datasets demonstrate competitive performance compared to baselines. It also shows potential for accurate predictions, particularly in imbalanced-distribution graphs.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106828"},"PeriodicalIF":6.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graphs are ubiquitous in real-world applications, such as computation graphs and social networks. Partitioning large graphs into smaller, balanced partitions is often essential, with the bi-objective graph partitioning problem aiming to minimize both the “cut” across partitions and the imbalance in partition sizes. However, existing heuristic methods face scalability challenges or overlook partition balance, leading to suboptimal results. Recent deep learning approaches, while promising, typically focus only on node-level features and lack a truly end-to-end framework, resulting in limited performance. In this paper, we introduce a novel method based on graph neural networks (GNNs) that leverages multilevel graph features and addresses the problem end-to-end through a bi-objective formulation. Our approach explores node-, local-, and global-level features, and introduces a well-bounded bi-objective function that minimizes the cut while ensuring partition-wise balance across all partitions. Additionally, we propose a GNN-based deep model incorporating a operator, allowing the model to optimize partitions in a fully end-to-end manner. Experimental results on 12 datasets across various applications and scales demonstrate that our method significantly improves both partitioning quality and scalability compared to existing bi-objective and deep graph partitioning baselines.
{"title":"An end-to-end bi-objective approach to deep graph partitioning","authors":"Pengcheng Wei , Yuan Fang , Zhihao Wen , Zheng Xiao , Binbin Chen","doi":"10.1016/j.neunet.2024.106823","DOIUrl":"10.1016/j.neunet.2024.106823","url":null,"abstract":"<div><div>Graphs are ubiquitous in real-world applications, such as computation graphs and social networks. Partitioning large graphs into smaller, balanced partitions is often essential, with the bi-objective graph partitioning problem aiming to minimize both the “cut” across partitions and the imbalance in partition sizes. However, existing heuristic methods face scalability challenges or overlook partition balance, leading to suboptimal results. Recent deep learning approaches, while promising, typically focus only on node-level features and lack a truly end-to-end framework, resulting in limited performance. In this paper, we introduce a novel method based on graph neural networks (GNNs) that leverages multilevel graph features and addresses the problem end-to-end through a bi-objective formulation. Our approach explores node-, local-, and global-level features, and introduces a well-bounded bi-objective function that minimizes the cut while ensuring <em>partition-wise</em> balance across all partitions. Additionally, we propose a GNN-based deep model incorporating a <span><math><mo>Hardmax</mo></math></span> operator, allowing the model to optimize partitions in a fully end-to-end manner. Experimental results on 12 datasets across various applications and scales demonstrate that our method significantly improves both partitioning quality and scalability compared to existing bi-objective and deep graph partitioning baselines.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106823"},"PeriodicalIF":6.0,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-20DOI: 10.1016/j.neunet.2024.106836
Shuaicong Hu , Yanan Wang , Jian Liu , Zhaoqiang Cui , Cuiwei Yang , Zhifeng Yao , Junbo Ge
Obstructive sleep apnea (OSA) is a common sleep breathing disorder and timely diagnosis helps to avoid the serious medical expenses caused by related complications. Existing deep learning (DL)-based methods primarily focus on single-modal models, which cannot fully mine task-related representations. This paper develops a modality fusion representation enhancement (MFRE) framework adaptable to flexible modality fusion types with the objective of improving OSA diagnostic performance, and providing quantitative evidence for clinical diagnostic modality selection. The proposed parallel information bottleneck modality fusion network (IPCT-Net) can extract local-global multi-view representations and eliminate redundant information in modality fusion representations through branch sharing mechanisms. We utilize large-scale real-world home sleep apnea test (HSAT) multimodal data to comprehensively evaluate relevant modality fusion types. Extensive experiments demonstrate that the proposed method significantly outperforms existing methods in terms of participant numbers and OSA diagnostic performance. The proposed MFRE framework delves into modality fusion in OSA diagnosis and contributes to enhancing the screening performance of artificial intelligence (AI)-assisted diagnosis for OSA.
阻塞性睡眠呼吸暂停（OSA）是一种常见的睡眠呼吸障碍，及时诊断有助于避免相关并发症造成的严重医疗费用。现有的基于深度学习（DL）的方法主要侧重于单模态模型，无法充分挖掘与任务相关的表征。本文开发了一种适应灵活模态融合类型的模态融合表征增强（MFRE）框架，旨在提高 OSA 诊断性能，为临床诊断模态选择提供定量证据。本文提出的并行信息瓶颈模态融合网络（IPCT-Net）可以提取局部-全局多视角表征，并通过分支共享机制消除模态融合表征中的冗余信息。我们利用大规模真实家庭睡眠呼吸暂停测试（HSAT）多模态数据，全面评估了相关模态融合类型。广泛的实验证明，所提出的方法在参与人数和 OSA 诊断性能方面明显优于现有方法。所提出的 MFRE 框架深入研究了 OSA 诊断中的模态融合，有助于提高人工智能辅助诊断 OSA 的筛查性能。
{"title":"IPCT-Net: Parallel information bottleneck modality fusion network for obstructive sleep apnea diagnosis","authors":"Shuaicong Hu , Yanan Wang , Jian Liu , Zhaoqiang Cui , Cuiwei Yang , Zhifeng Yao , Junbo Ge","doi":"10.1016/j.neunet.2024.106836","DOIUrl":"10.1016/j.neunet.2024.106836","url":null,"abstract":"<div><div>Obstructive sleep apnea (OSA) is a common sleep breathing disorder and timely diagnosis helps to avoid the serious medical expenses caused by related complications. Existing deep learning (DL)-based methods primarily focus on single-modal models, which cannot fully mine task-related representations. This paper develops a modality fusion representation enhancement (MFRE) framework adaptable to flexible modality fusion types with the objective of improving OSA diagnostic performance, and providing quantitative evidence for clinical diagnostic modality selection. The proposed parallel information bottleneck modality fusion network (IPCT-Net) can extract local-global multi-view representations and eliminate redundant information in modality fusion representations through branch sharing mechanisms. We utilize large-scale real-world home sleep apnea test (HSAT) multimodal data to comprehensively evaluate relevant modality fusion types. Extensive experiments demonstrate that the proposed method significantly outperforms existing methods in terms of participant numbers and OSA diagnostic performance. The proposed MFRE framework delves into modality fusion in OSA diagnosis and contributes to enhancing the screening performance of artificial intelligence (AI)-assisted diagnosis for OSA.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106836"},"PeriodicalIF":6.0,"publicationDate":"2024-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号