Neurocomputing最新文献

英文中文

LAR-Pose: Lightweight human pose estimation with adaptive regression loss

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129777

Xudong Lou , Xin Lin , Henan Zeng , Xiangxian Zhu

In this paper, LAR-Pose, a lightweight, high-resolution network for human pose estimation driven by adaptive regression loss is proposed and experimentally demonstrated based on MS COCO and MPII. The architecture of the LAR-Pose comprises two main components. One is a lightweight high-resolution backbone network, which utilizes a parallel high-resolution architecture with conditional channel weighting block to reduce the model size and computational complexity. The other is a dynamic residual refinement network, which calculates residuals from pseudo-heatmaps and scaling factors, improving training concentration for consistent distribution estimation, rather than predicting coordinates or heatmaps directly. Specific coordinates are derived through integral heatmap regression, effectively minimizing quantization errors. Our adaptive regression loss, which uses a flow model to fit the distribution of residuals in real-time, provides more sensitive parameter feedback than conventional heatmap loss, ensuring differentiability and continuity during backpropagation while enhancing performance. With a relatively small parameter scale, LAR-Pose achieves an AP of 73.5 on MS COCO and a PCKh of 90.9 on MPII, while the results outperform most advanced small networks and approach the performance of large networks.

{"title":"LAR-Pose: Lightweight human pose estimation with adaptive regression loss","authors":"Xudong Lou , Xin Lin , Henan Zeng , Xiangxian Zhu","doi":"10.1016/j.neucom.2025.129777","DOIUrl":"10.1016/j.neucom.2025.129777","url":null,"abstract":"<div><div>In this paper, LAR-Pose, a lightweight, high-resolution network for human pose estimation driven by adaptive regression loss is proposed and experimentally demonstrated based on MS COCO and MPII. The architecture of the LAR-Pose comprises two main components. One is a lightweight high-resolution backbone network, which utilizes a parallel high-resolution architecture with conditional channel weighting block to reduce the model size and computational complexity. The other is a dynamic residual refinement network, which calculates residuals from pseudo-heatmaps and scaling factors, improving training concentration for consistent distribution estimation, rather than predicting coordinates or heatmaps directly. Specific coordinates are derived through integral heatmap regression, effectively minimizing quantization errors. Our adaptive regression loss, which uses a flow model to fit the distribution of residuals in real-time, provides more sensitive parameter feedback than conventional heatmap loss, ensuring differentiability and continuity during backpropagation while enhancing performance. With a relatively small parameter scale, LAR-Pose achieves an AP of 73.5 on MS COCO and a PCKh of 90.9 on MPII, while the results outperform most advanced small networks and approach the performance of large networks.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"633 ","pages":"Article 129777"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143529536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Event-based video reconstruction via attention-based recurrent network

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129776

Wenwen Ma, Shanxing Ma, Pieter Meiresone, Gianni Allebosch, Wilfried Philips, Jan Aelterman

Event cameras are novel sensors that capture brightness changes in the form of asynchronous events rather than intensity frames, offering unique advantages such as high dynamic range, high temporal resolution, and no motion blur. However, the sparse, asynchronous nature of event data poses significant challenges for visual perception, limiting compatibility with conventional computer vision algorithms that rely on dense, continuous frames. Event-based video reconstruction has emerged as a promising solution, though existing methods still face challenges in capturing fine-grained details and enhancing contrast. This paper presents a novel approach to video reconstruction from asynchronous event streams, leveraging the unique properties of event data to produce high-quality video. Our method integrates channel and pixel attention mechanisms to focus on essential features and incorporates deformable convolutions and adaptive mix-up operations to provide flexible receptive fields and dynamic fusion across down-sampling and up-sampling layers. Experimental results on multiple real-world event datasets demonstrate that our approach outperforms comparable methods trained on the same dataset, achieving superior video quality from pure event data. We also demonstrate the capability of our method for high dynamic range reconstruction and color video reconstruction using an event camera equipped with a Bayer filter.

{"title":"Event-based video reconstruction via attention-based recurrent network","authors":"Wenwen Ma, Shanxing Ma, Pieter Meiresone, Gianni Allebosch, Wilfried Philips, Jan Aelterman","doi":"10.1016/j.neucom.2025.129776","DOIUrl":"10.1016/j.neucom.2025.129776","url":null,"abstract":"<div><div>Event cameras are novel sensors that capture brightness changes in the form of asynchronous events rather than intensity frames, offering unique advantages such as high dynamic range, high temporal resolution, and no motion blur. However, the sparse, asynchronous nature of event data poses significant challenges for visual perception, limiting compatibility with conventional computer vision algorithms that rely on dense, continuous frames. Event-based video reconstruction has emerged as a promising solution, though existing methods still face challenges in capturing fine-grained details and enhancing contrast. This paper presents a novel approach to video reconstruction from asynchronous event streams, leveraging the unique properties of event data to produce high-quality video. Our method integrates channel and pixel attention mechanisms to focus on essential features and incorporates deformable convolutions and adaptive mix-up operations to provide flexible receptive fields and dynamic fusion across down-sampling and up-sampling layers. Experimental results on multiple real-world event datasets demonstrate that our approach outperforms comparable methods trained on the same dataset, achieving superior video quality from pure event data. We also demonstrate the capability of our method for high dynamic range reconstruction and color video reconstruction using an event camera equipped with a Bayer filter.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"632 ","pages":"Article 129776"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143509214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Second-order consensus of matrix-weighted switched multiagent systems

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129755

Suoxia Miao , Housheng Su

Due to the switching characteristics of many practical multi-agent systems, such as automatic speed control systems and hybrid quadcopters, for multidimensional individuals, combined with practical complexity, matrix-weighted switching dynamics are needed to model them. This paper considers the consensus issues of second-order switched multi-agent systems on matrix-weighted undirected and directed networks. A new matrix-weighted control algorithm suitable for both CT and DT subsystems is proposed. Under the proposed algorithms, based on variable transformation, matrix theory and stability theory, the consensus criteria are established for undirected and directed switched multi-agent networks that rely on the eigenvalues of the network and coupling gains, respectively. This also indicates that the matrix-weights and coupling gains have a significant impact on switched matrix-weighted consensus. Finally, through simulations, the validity of the obtained results of this essay are verified.

引用次数: 0

Attribute graph anomaly detection utilizing memory networks enhanced by multi-embedding comparison 利用多嵌入比较增强的记忆网络进行属性图异常检测

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129762

Lianming Zhang, Baolin Wu, Pingping Dong

In complex attribute networks, accurately pinpointing anomalous nodes is vital for grasping network behavior and safeguarding network security. Traditional anomaly detection methods often struggle to fully harness the intricate relationships that underpin attributes and structures, thus curbing their practical effectiveness. To transcend this limitation, we introduce a novel graph anomaly detection model that harmoniously integrates node attributes and structural information. Our model employs multi-embedding contrast modules, coupled with memory network enhancements, to pinpoint anomalous nodes. Precisely, we crafted a multi-embedding contrast module to encode the attributes and structures inherent within nodes, generating a multitude of embedding representations. By scrutinizing the discrepancies between these representations, our model adeptly identifies nodes that deviate from attribute and structural consistency, indicating anomalies. Furthermore, we incorporate a memory network to reconstruct node attributes, thereby enhancing the attribute decoding process while preserving the straightforwardness of structural decoding. To validate our method, we conducted extensive experiments on five authoritative public graph datasets, comparing various graph anomaly detection methods using rigorous metrics such as AUC, precision, and recall. The experimental results unequivocally demonstrate that our proposed method surpasses current state-of-the-art techniques in detecting anomalous nodes within graphs, solidly validating its efficacy.

{"title":"Attribute graph anomaly detection utilizing memory networks enhanced by multi-embedding comparison","authors":"Lianming Zhang, Baolin Wu, Pingping Dong","doi":"10.1016/j.neucom.2025.129762","DOIUrl":"10.1016/j.neucom.2025.129762","url":null,"abstract":"<div><div>In complex attribute networks, accurately pinpointing anomalous nodes is vital for grasping network behavior and safeguarding network security. Traditional anomaly detection methods often struggle to fully harness the intricate relationships that underpin attributes and structures, thus curbing their practical effectiveness. To transcend this limitation, we introduce a novel graph anomaly detection model that harmoniously integrates node attributes and structural information. Our model employs multi-embedding contrast modules, coupled with memory network enhancements, to pinpoint anomalous nodes. Precisely, we crafted a multi-embedding contrast module to encode the attributes and structures inherent within nodes, generating a multitude of embedding representations. By scrutinizing the discrepancies between these representations, our model adeptly identifies nodes that deviate from attribute and structural consistency, indicating anomalies. Furthermore, we incorporate a memory network to reconstruct node attributes, thereby enhancing the attribute decoding process while preserving the straightforwardness of structural decoding. To validate our method, we conducted extensive experiments on five authoritative public graph datasets, comparing various graph anomaly detection methods using rigorous metrics such as AUC, precision, and recall. The experimental results unequivocally demonstrate that our proposed method surpasses current state-of-the-art techniques in detecting anomalous nodes within graphs, solidly validating its efficacy.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"633 ","pages":"Article 129762"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143526702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Transref: Multi-scale reference embedding transformer for reference-guided image inpainting

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129749

Taorong Liu , Liang Liao , Delin Chen , Jing Xiao , Zheng Wang , Chia-Wen Lin , Shin’ichi Satoh

Image inpainting for completing complicated semantic environments and diverse hole patterns of corrupted images is challenging even for state-of-the-art learning-based inpainting methods trained on large-scale data. A reference image capturing the same scene of a corrupted image offers informative guidance for completing the corrupted image as it shares similar texture and structure priors to that of the holes of the corrupted image. In this work, we propose a Transformer-based encoder–decoder network for Reference-guided image inpainting, named TransRef. Specifically, the guidance is conducted progressively through a reference embedding procedure, in which the referencing features are subsequently aligned and fused with the features of the corrupted image. For precise utilization of the reference features for guidance, a reference-patch alignment (Ref-PA) module is proposed to align the patch features of the reference and corrupted images and harmonize their style differences, while a reference-patch transformer (Ref-PT) module is proposed to refine the embedded reference feature. Moreover, to facilitate the research of reference-guided image restoration tasks, we construct a publicly accessible benchmark dataset containing 50K pairs of input and reference images. Both quantitative and qualitative evaluations demonstrate the efficacy of the reference information and the proposed method over the state-of-the-art methods in completing complex holes. Code and dataset can be accessed at: https://github.com/Cameltr/TransRef.

{"title":"Transref: Multi-scale reference embedding transformer for reference-guided image inpainting","authors":"Taorong Liu , Liang Liao , Delin Chen , Jing Xiao , Zheng Wang , Chia-Wen Lin , Shin’ichi Satoh","doi":"10.1016/j.neucom.2025.129749","DOIUrl":"10.1016/j.neucom.2025.129749","url":null,"abstract":"<div><div>Image inpainting for completing complicated semantic environments and diverse hole patterns of corrupted images is challenging even for state-of-the-art learning-based inpainting methods trained on large-scale data. A reference image capturing the same scene of a corrupted image offers informative guidance for completing the corrupted image as it shares similar texture and structure priors to that of the holes of the corrupted image. In this work, we propose a <strong>Trans</strong>former-based encoder–decoder network for <strong>Ref</strong>erence-guided image inpainting, named <strong>TransRef</strong>. Specifically, the guidance is conducted progressively through a reference embedding procedure, in which the referencing features are subsequently aligned and fused with the features of the corrupted image. For precise utilization of the reference features for guidance, a reference-patch alignment (Ref-PA) module is proposed to align the patch features of the reference and corrupted images and harmonize their style differences, while a reference-patch transformer (Ref-PT) module is proposed to refine the embedded reference feature. Moreover, to facilitate the research of reference-guided image restoration tasks, we construct a publicly accessible benchmark dataset containing 50K pairs of input and reference images. Both quantitative and qualitative evaluations demonstrate the efficacy of the reference information and the proposed method over the state-of-the-art methods in completing complex holes. Code and dataset can be accessed at: <span><span>https://github.com/Cameltr/TransRef</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"632 ","pages":"Article 129749"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143509215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rehearsal-free continual few-shot relation extraction via contrastive weighted prompts

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129741

Fengqin Yang, Mengen Ren, Delu Kong, Shuhua Liu, Zhiguo Fu

The primary challenge in continual few-shot relation extraction is mitigating catastrophic forgetting. Prevailing strategies involve saving a set of samples in memory and replaying them. However, these methods pose privacy and data security concerns. To address this, we propose a novel rehearsal-free approach called Contrastive Weighted Prompt (CWP). This approach categorizes learnable prompts into task-generic and task-specific prompts. Task-generic prompts are shared across all tasks and are injected into the higher layers of the BERT encoder to capture general task knowledge. Task-specific prompts are generated by weighting all the prompts in a task-specific prompt pool based on their relevance to individual samples. These task-specific prompts are injected into the lower layers of BERT to extract task-specific knowledge. Task-generic prompts retain knowledge from prior tasks, while task-specific prompts reduce mutual interference among tasks and improve the relevance between prompts and individual samples. To further enhance the discriminability of the prompt embeddings for samples belonging to different relations, we introduced a relation-aware contrastive learning strategy. Experimental results on two standard datasets indicate that the proposed method outperforms baseline methods and demonstrates superiority in mitigating catastrophic forgetting.

{"title":"Rehearsal-free continual few-shot relation extraction via contrastive weighted prompts","authors":"Fengqin Yang, Mengen Ren, Delu Kong, Shuhua Liu, Zhiguo Fu","doi":"10.1016/j.neucom.2025.129741","DOIUrl":"10.1016/j.neucom.2025.129741","url":null,"abstract":"<div><div>The primary challenge in continual few-shot relation extraction is mitigating catastrophic forgetting. Prevailing strategies involve saving a set of samples in memory and replaying them. However, these methods pose privacy and data security concerns. To address this, we propose a novel rehearsal-free approach called Contrastive Weighted Prompt (CWP). This approach categorizes learnable prompts into task-generic and task-specific prompts. Task-generic prompts are shared across all tasks and are injected into the higher layers of the BERT encoder to capture general task knowledge. Task-specific prompts are generated by weighting all the prompts in a task-specific prompt pool based on their relevance to individual samples. These task-specific prompts are injected into the lower layers of BERT to extract task-specific knowledge. Task-generic prompts retain knowledge from prior tasks, while task-specific prompts reduce mutual interference among tasks and improve the relevance between prompts and individual samples. To further enhance the discriminability of the prompt embeddings for samples belonging to different relations, we introduced a relation-aware contrastive learning strategy. Experimental results on two standard datasets indicate that the proposed method outperforms baseline methods and demonstrates superiority in mitigating catastrophic forgetting.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"633 ","pages":"Article 129741"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143520417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SHoTGCN: Spatial high-order temporal GCN for skeleton-based action recognition

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129697

Qiyu Liu , Ying Wu , Bicheng Li , Yuxin Ma , Hanling Li , Yong Yu

Action recognition algorithms that leverage human skeleton motion data are highly attractive due to their robustness and high information density. Currently, the majority of algorithms in this domain employ graph convolutional neural networks (GCNs). However, these algorithms often neglect the extraction of high-order features. To address this limitation, we propose a novel approach called the Spatial High-Order Temporal Graph Convolution Network (SHoTGCN), designed to evaluate the impact of high-order features on human action recognition. Our method begins by deriving high-order features from human skeleton time series data through temporal interactions. Utilizing these high-order features significantly improves the algorithm’s ability to recognize human actions. Moreover, we found that the traditional feature extraction method, which employs Depthwise Convolution (DWConv) with a single 2D convolution, is suboptimal compared to a multibranch structure for feature extraction. To address this, we introduce a structure re-parameterization technique with DWConv, termed Rep-tDWConv, to enhance feature extraction. By integrating the Exponential Moving Average (EMA) model during the model fusion process, our proposed model achieves state-of-the-art (SOTA) performance, with accuracies of 90.4% and 92.0% on the XSub and XSet splits of the NTU RGB+D 120 dataset, respectively.

{"title":"SHoTGCN: Spatial high-order temporal GCN for skeleton-based action recognition","authors":"Qiyu Liu , Ying Wu , Bicheng Li , Yuxin Ma , Hanling Li , Yong Yu","doi":"10.1016/j.neucom.2025.129697","DOIUrl":"10.1016/j.neucom.2025.129697","url":null,"abstract":"<div><div>Action recognition algorithms that leverage human skeleton motion data are highly attractive due to their robustness and high information density. Currently, the majority of algorithms in this domain employ graph convolutional neural networks (GCNs). However, these algorithms often neglect the extraction of high-order features. To address this limitation, we propose a novel approach called the Spatial High-Order Temporal Graph Convolution Network (SHoTGCN), designed to evaluate the impact of high-order features on human action recognition. Our method begins by deriving high-order features from human skeleton time series data through temporal interactions. Utilizing these high-order features significantly improves the algorithm’s ability to recognize human actions. Moreover, we found that the traditional feature extraction method, which employs Depthwise Convolution (DWConv) with a single 2D convolution, is suboptimal compared to a multibranch structure for feature extraction. To address this, we introduce a structure re-parameterization technique with DWConv, termed Rep-tDWConv, to enhance feature extraction. By integrating the Exponential Moving Average (EMA) model during the model fusion process, our proposed model achieves state-of-the-art (SOTA) performance, with accuracies of 90.4% and 92.0% on the XSub and XSet splits of the NTU RGB+D 120 dataset, respectively.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"632 ","pages":"Article 129697"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143508801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An O(1/k) algorithm for multi-agent optimization with inequality constraints

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129770

Peng Li , Yiyi Zhao , Jiangping Hu , Jiangtao Ji

This paper presents a discrete-time solution algorithm for a constrained multi-agent optimization problem with inequality constraints. Its aim is to seek a solution to minimize the sum of all the agents’ objective functions while satisfy each agent’s local set constraint and nonlinear inequality constraints. Assume that agents’ local constraints are heterogeneous and all the objective functions are convex and continuous, but they may not be differentiable. Similar to the distributed alternating direction method of multipliers (ADMM) algorithm, the designed algorithm can solve the multi-agent optimization problem in a distributed manner and has a fast

O (1 / k)

convergence rate. Moreover, it can deal with the nonlinear constraints, which cannot be handled by distributed ADMM algorithm. Finally, the proposed algorithm is applied to solve a robust linear regression problem, a lasso problem and a decentralized joint flow and power control problem with inequality constraints, respectively and thus the effectiveness of the proposed algorithm is verified.

{"title":"An O(1/k) algorithm for multi-agent optimization with inequality constraints","authors":"Peng Li , Yiyi Zhao , Jiangping Hu , Jiangtao Ji","doi":"10.1016/j.neucom.2025.129770","DOIUrl":"10.1016/j.neucom.2025.129770","url":null,"abstract":"<div><div>This paper presents a discrete-time solution algorithm for a constrained multi-agent optimization problem with inequality constraints. Its aim is to seek a solution to minimize the sum of all the agents’ objective functions while satisfy each agent’s local set constraint and nonlinear inequality constraints. Assume that agents’ local constraints are heterogeneous and all the objective functions are convex and continuous, but they may not be differentiable. Similar to the distributed alternating direction method of multipliers (ADMM) algorithm, the designed algorithm can solve the multi-agent optimization problem in a distributed manner and has a fast <span><math><mrow><mi>O</mi><mrow><mo>(</mo><mn>1</mn><mo>/</mo><mi>k</mi><mo>)</mo></mrow></mrow></math></span> convergence rate. Moreover, it can deal with the nonlinear constraints, which cannot be handled by distributed ADMM algorithm. Finally, the proposed algorithm is applied to solve a robust linear regression problem, a lasso problem and a decentralized joint flow and power control problem with inequality constraints, respectively and thus the effectiveness of the proposed algorithm is verified.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"632 ","pages":"Article 129770"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143509211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AA-mDLAM: An accelerated ADMM-based framework for training deep neural networks AA-mDLAM：基于 ADMM 的加速深度神经网络训练框架

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129744

Zeinab Ebrahimi , Gustavo Batista , Mohammad Deghat

Stochastic gradient descent (SGD) and its many variants are the widespread optimization algorithms for training deep neural networks. However, SGD suffers from inevitable drawbacks, including vanishing gradients, lack of theoretical guarantees, and substantial sensitivity to input. The Alternating Direction Method of Multipliers (ADMM) has been proposed to address these shortcomings as an effective alternative to the gradient-based methods. It has been successfully employed for training deep neural networks. However, ADMM-based optimizers have a slow convergence rate. This paper proposes an accelerated framework for training deep neural networks, termed AA-mDLAM, which integrates Anderson acceleration within an Alternating Minimization approach inspired by ADMM to tackle this drawback. The main intention of the AA-mDLAM algorithm is to employ Anderson acceleration to alternating minimization by considering it as a fixed-point iteration and attaining a nearly quadratic convergence rate. We verify the effectiveness and efficiency of the proposed AA-mDLAM algorithm by conducting extensive experiments on seven benchmark datasets contrary to other state-of-the-art optimizers.

{"title":"AA-mDLAM: An accelerated ADMM-based framework for training deep neural networks","authors":"Zeinab Ebrahimi , Gustavo Batista , Mohammad Deghat","doi":"10.1016/j.neucom.2025.129744","DOIUrl":"10.1016/j.neucom.2025.129744","url":null,"abstract":"<div><div>Stochastic gradient descent (SGD) and its many variants are the widespread optimization algorithms for training deep neural networks. However, SGD suffers from inevitable drawbacks, including vanishing gradients, lack of theoretical guarantees, and substantial sensitivity to input. The Alternating Direction Method of Multipliers (ADMM) has been proposed to address these shortcomings as an effective alternative to the gradient-based methods. It has been successfully employed for training deep neural networks. However, ADMM-based optimizers have a slow convergence rate. This paper proposes an accelerated framework for training deep neural networks, termed AA-mDLAM, which integrates Anderson acceleration within an Alternating Minimization approach inspired by ADMM to tackle this drawback. The main intention of the AA-mDLAM algorithm is to employ Anderson acceleration to alternating minimization by considering it as a fixed-point iteration and attaining a nearly quadratic convergence rate. We verify the effectiveness and efficiency of the proposed AA-mDLAM algorithm by conducting extensive experiments on seven benchmark datasets contrary to other state-of-the-art optimizers.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"633 ","pages":"Article 129744"},"PeriodicalIF":5.5,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143529542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A variable-gain fixed-time convergent neurodynamic network for time-variant quadratic programming under unknown noises

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2025-02-25 DOI: 10.1016/j.neucom.2025.129778

Biao Song , Tinghe Hong , Weibing Li , Gang Chen , Yongping Pan , Kai Huang

This article proposes a variable-gain fixed-time convergent and noise-tolerant error-dynamics based neurodynamic network (VGFxTNT-EDNN) to solve time-varying quadratic programming problems, while being robust to unknown noises. Unlike existing finite-time convergent EDNNs, the newly designed VGFxTNT-EDNN guarantees fixed-time convergence by dynamically adjusting its variable parameters. Moreover, the VGFxTNT-EDNN effectively handles unknown noise, addressing a limitation of existing fixed-time or predefined-time convergent models, which typically assume that the noise is known. Theoretical analysis utilizing Lyapunov theory proves that the VGFxTNT-EDNN possesses fixed-time convergence and robustness properties. Numerical validations demonstrate superior noise tolerance and fixed-time convergence of the VGFxTNT-EDNN, as compared with the existing models. Finally, a path-tracking experiment is conducted by utilizing a Franka Emika Panda robot to verify the practicality of the VGFxTNT-EDNN.

引用次数: 0

首页上一页

下一页尾页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Neurocomputing

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀