Pub Date : 2026-02-03DOI: 10.1016/j.neunet.2026.108678
Xiaopeng Xu, Yadong Zhao, Dong Wang
In partially observable multi-agent tasks, communication modules need to be trained to supplement information for decision-making, and decision-making modules need to be trained to analyze received information. This leads to their coupling during the process of multi-agent reinforcement learning. To alleviate the difficulty from the coupling, two transfer-based frameworks are proposed. The Simple Transfer Framework converts a difficult reinforcement learning into an easy one and a supervised learning. The Decomposition and Transfer Framework further decomposes this supervised learning into two simpler ones, based on the decomposition of communication-based individual Q-values. They are decomposed into local individual Q-values and their deviations. Based on this framework, the mutual information neural estimation is introduced to train message generators independently. The proposed frameworks are verified by training two communication-based architectures. They outperform direct reinforcement learning algorithms across various team sizes tested in the Drone Fight. The independent training method effectively facilitates the transfer of the deviations.
{"title":"Decomposition and transfer of individual Q-values for decision-making of multi-agent reinforcement learning with communication","authors":"Xiaopeng Xu, Yadong Zhao, Dong Wang","doi":"10.1016/j.neunet.2026.108678","DOIUrl":"10.1016/j.neunet.2026.108678","url":null,"abstract":"<div><div>In partially observable multi-agent tasks, communication modules need to be trained to supplement information for decision-making, and decision-making modules need to be trained to analyze received information. This leads to their coupling during the process of multi-agent reinforcement learning. To alleviate the difficulty from the coupling, two transfer-based frameworks are proposed. The Simple Transfer Framework converts a difficult reinforcement learning into an easy one and a supervised learning. The Decomposition and Transfer Framework further decomposes this supervised learning into two simpler ones, based on the decomposition of communication-based individual Q-values. They are decomposed into local individual Q-values and their deviations. Based on this framework, the mutual information neural estimation is introduced to train message generators independently. The proposed frameworks are verified by training two communication-based architectures. They outperform direct reinforcement learning algorithms across various team sizes tested in the Drone Fight. The independent training method effectively facilitates the transfer of the deviations.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108678"},"PeriodicalIF":6.3,"publicationDate":"2026-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146167535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-03DOI: 10.1016/j.neunet.2026.108674
Kang Liu , Yuqi Zhang , Shunzhi Yang , Chang-Dong Wang , Yunwen Chen , Xiaowen Ma , Zhenhua Huang
Existing graph knowledge distillation methods suffer from limited absorption of the teacher’s “dark knowledge” because they rely on simple logit alignment, which often causes overfitting or incomplete capture of underlying patterns. Additionally, relying on a single perspective severely restricts the student’s learning effectiveness and generalization ability. To address these issues, we develop a novel Multiple Interpretation Ensemble Distillation (MIED) method. It constructs a multi-interpreter composed of multiple single-layer MLPs for the student, termed the Student Interpretation (SI) component, to interpret knowledge from diversified outputs, thus avoiding representational bias from a single student output. Based on this, it introduces two effective strategies, i.e., Hybrid Sampling and Hierarchical Update. The former employs different sampling strategies for the outputs of the teacher and student (including the SI component). Specifically, the teacher’s output adopts a percentage random sampler, while the outputs of the student and SI component both leverage a positive-negative sampler. With this design, MIED can facilitate better coordination of sample selection and the learning process among the teacher, student, and SI component. The latter updates the parameters of the last layer in the student using the exponential moving average of the fused parameters of the SI component, while the parameters of other layers are updated via a regular optimizer. This enhances the robustness and generalization performance of MIED. Extensive experiments on seven real-world public datasets demonstrate that MIED outperforms existing methods in node classification tasks, resulting in an average improvement of 5.56% over GCN and 27.43% over MLP, respectively. Moreover, compared with directly using multiple students (where the number is consistent with the number of layers in the SI component), MIED achieves improvements approximately 6.00% in time, 50.00% in space, and 0.20% in accuracy. These results indicate that MIED is scalable and generalizable, and exhibits robustness on complex samples.
{"title":"Multiple interpretation ensemble distillation for graph neural networks","authors":"Kang Liu , Yuqi Zhang , Shunzhi Yang , Chang-Dong Wang , Yunwen Chen , Xiaowen Ma , Zhenhua Huang","doi":"10.1016/j.neunet.2026.108674","DOIUrl":"10.1016/j.neunet.2026.108674","url":null,"abstract":"<div><div>Existing graph knowledge distillation methods suffer from limited absorption of the teacher’s “dark knowledge” because they rely on simple logit alignment, which often causes overfitting or incomplete capture of underlying patterns. Additionally, relying on a single perspective severely restricts the student’s learning effectiveness and generalization ability. To address these issues, we develop a novel Multiple Interpretation Ensemble Distillation (MIED) method. It constructs a multi-interpreter composed of multiple single-layer MLPs for the student, termed the Student Interpretation (SI) component, to interpret knowledge from diversified outputs, thus avoiding representational bias from a single student output. Based on this, it introduces two effective strategies, i.e., Hybrid Sampling and Hierarchical Update. The former employs different sampling strategies for the outputs of the teacher and student (including the SI component). Specifically, the teacher’s output adopts a percentage random sampler, while the outputs of the student and SI component both leverage a positive-negative sampler. With this design, MIED can facilitate better coordination of sample selection and the learning process among the teacher, student, and SI component. The latter updates the parameters of the last layer in the student using the exponential moving average of the fused parameters of the SI component, while the parameters of other layers are updated via a regular optimizer. This enhances the robustness and generalization performance of MIED. Extensive experiments on seven real-world public datasets demonstrate that MIED outperforms existing methods in node classification tasks, resulting in an average improvement of 5.56% over GCN and 27.43% over MLP, respectively. Moreover, compared with directly using multiple students (where the number is consistent with the number of layers in the SI component), MIED achieves improvements approximately 6.00% in time, 50.00% in space, and 0.20% in accuracy. These results indicate that MIED is scalable and generalizable, and exhibits robustness on complex samples.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108674"},"PeriodicalIF":6.3,"publicationDate":"2026-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146167504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-03DOI: 10.1016/j.neunet.2026.108680
Ben Liang , Yuan Liu , Chao Sui , Yihong Wang , Lin Xiao , Xiubao Sui , Qian Chen
With the advancement of high-precision remote sensing equipment and precision measurement technology, object detection based on remote sensing images (RSIs) has been widely used in military and civilian fields. Different from traditional general-purpose environments, remote sensing presents unique challenges that significantly complicate the detection process. Specifically: (1) RSIs cover extensive monitoring areas, resulting in complex and textured backgrounds; and (2) objects often exhibit cluttered distributions, small sizes, and considerable scale variations across categories. To effectively address these challenges, we propose a Multi-Scale Pattern-Aware Task-Gating Network (MPTNet) for remote sensing object detection. First, we design a Multi-Scale Pattern-Aware Network (MPNet) backbone that employs a small and large kernel convolutional complementary strategy to capture both large-scale and small-scale spatial patterns, yielding more comprehensive semantic features. Next, we introduce a Multi-Head Cross-Space Encoder (MCE) that improves semantic fusion and spatial representation across hierarchical levels. By combining a multi-head mechanism with directional one-dimensional strip convolutions, MCE enhances spatial sensitivity at the pixel level, thus improving object localization in densely textured scenes. To harmonize cross-task synergy, we propose a Dynamic Task-Gating (DTG) head that adaptively recalibrates spatial feature representations between classification and localization branches. Extensive experimental validations on three publicly available datasets, including VisDrone, DIOR, and COCO-mini, demonstrate that our method achieves excellent performance, obtaining AP50 scores of 43.3%, 80.6%, and 49.5%, respectively.
{"title":"Multi-Scale pattern-Aware task-Gating network for aerial small object detection","authors":"Ben Liang , Yuan Liu , Chao Sui , Yihong Wang , Lin Xiao , Xiubao Sui , Qian Chen","doi":"10.1016/j.neunet.2026.108680","DOIUrl":"10.1016/j.neunet.2026.108680","url":null,"abstract":"<div><div>With the advancement of high-precision remote sensing equipment and precision measurement technology, object detection based on remote sensing images (RSIs) has been widely used in military and civilian fields. Different from traditional general-purpose environments, remote sensing presents unique challenges that significantly complicate the detection process. Specifically: (1) RSIs cover extensive monitoring areas, resulting in complex and textured backgrounds; and (2) objects often exhibit cluttered distributions, small sizes, and considerable scale variations across categories. To effectively address these challenges, we propose a Multi-Scale Pattern-Aware Task-Gating Network (MPTNet) for remote sensing object detection. First, we design a Multi-Scale Pattern-Aware Network (MPNet) backbone that employs a small and large kernel convolutional complementary strategy to capture both large-scale and small-scale spatial patterns, yielding more comprehensive semantic features. Next, we introduce a Multi-Head Cross-Space Encoder (MCE) that improves semantic fusion and spatial representation across hierarchical levels. By combining a multi-head mechanism with directional one-dimensional strip convolutions, MCE enhances spatial sensitivity at the pixel level, thus improving object localization in densely textured scenes. To harmonize cross-task synergy, we propose a Dynamic Task-Gating (DTG) head that adaptively recalibrates spatial feature representations between classification and localization branches. Extensive experimental validations on three publicly available datasets, including VisDrone, DIOR, and COCO-mini, demonstrate that our method achieves excellent performance, obtaining AP<sub>50</sub> scores of 43.3%, 80.6%, and 49.5%, respectively.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108680"},"PeriodicalIF":6.3,"publicationDate":"2026-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146144671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-02DOI: 10.1016/j.neunet.2026.108681
Filippo Aglietti , Francesco Della Santa , Andrea Piano , Virginia Aglietti
We propose Gradient-Informed Neural Networks (gradinn s), a methodology that can be used to efficiently approximate a wide range of functions in low-data regimes, when only general prior beliefs are available, a condition that is often encountered in complex engineering problems.
gradinn s incorporate prior beliefs about the first-order derivatives of the target function to constrain the behavior of its gradient, thus implicitly shaping it, without requiring explicit access to the target function’s derivatives. This is achieved by using two Neural Networks: one modeling the target function and a second, auxiliary network expressing the prior beliefs about the first-order derivatives (e.g., smoothness, oscillations, etc.). A customized loss function enables the training of the first network while enforcing gradient constraints derived from the auxiliary network; at the same time, it allows these constraints to be relaxed in accordance with the training data. Numerical experiments demonstrate the advantages of gradinn s, particularly in low-data regimes, with results showing strong performance compared to standard Neural Networks across the tested scenarios, including synthetic benchmark functions and real-world engineering tasks.
{"title":"Gradient-informed neural networks: Embedding prior beliefs for learning in low-data scenarios","authors":"Filippo Aglietti , Francesco Della Santa , Andrea Piano , Virginia Aglietti","doi":"10.1016/j.neunet.2026.108681","DOIUrl":"10.1016/j.neunet.2026.108681","url":null,"abstract":"<div><div>We propose Gradient-Informed Neural Networks (<span>g</span>rad<span>inn</span> s), a methodology that can be used to efficiently approximate a wide range of functions in low-data regimes, when only general prior beliefs are available, a condition that is often encountered in complex engineering problems.</div><div><span>g</span>rad<span>inn</span> s incorporate prior beliefs about the first-order derivatives of the target function to constrain the behavior of its gradient, thus implicitly shaping it, without requiring explicit access to the target function’s derivatives. This is achieved by using two Neural Networks: one modeling the target function and a second, auxiliary network expressing the prior beliefs about the first-order derivatives (e.g., smoothness, oscillations, etc.). A customized loss function enables the training of the first network while enforcing gradient constraints derived from the auxiliary network; at the same time, it allows these constraints to be relaxed in accordance with the training data. Numerical experiments demonstrate the advantages of <span>g</span>rad<span>inn</span> s, particularly in low-data regimes, with results showing strong performance compared to standard Neural Networks across the tested scenarios, including synthetic benchmark functions and real-world engineering tasks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108681"},"PeriodicalIF":6.3,"publicationDate":"2026-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146167562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-02DOI: 10.1016/j.neunet.2026.108682
Tianyu Hu , Renda Han , Liu Mao , Jing Chen , Xia Xie
Graph-level clustering aims to group graphs into distinct clusters based on shared structural characteristics or semantic similarities. However, existing graph-level clustering methods generally assume that the input graph structure is complete and overlook the problem of missing relationships that commonly exist in real-world scenarios. These unmodeled missing relationships will lead to the accumulation of structural information distortion during the graph representation learning process, significantly reducing the clustering performance. To this end, we propose a novel method, Structure-Missing Graph-Level Clustering Network (SMGCN), which includes a structure augmentation module LR-SEA, an Anchor Positioning Mechanism, and Joint Contrastive Optimization. Specifically, we first output augmented graphs based on low-rank matrix completion, perform cluster matching using the Hungarian algorithm to obtain anchors, and then force same clustering graphs to converge to the corresponding anchors in the embedding space. According to our research, this is the first time that the graph-level clustering task with missing relations is proposed, and the superiority of our method is demonstrated through experiments on five benchmark datasets, compared with the state-of-the-art methods. Our source codes are available at https://github.com/MrHuSN/SMGCN.
{"title":"Structure-missing graph-level clustering network","authors":"Tianyu Hu , Renda Han , Liu Mao , Jing Chen , Xia Xie","doi":"10.1016/j.neunet.2026.108682","DOIUrl":"10.1016/j.neunet.2026.108682","url":null,"abstract":"<div><div>Graph-level clustering aims to group graphs into distinct clusters based on shared structural characteristics or semantic similarities. However, existing graph-level clustering methods generally assume that the input graph structure is complete and overlook the problem of missing relationships that commonly exist in real-world scenarios. These unmodeled missing relationships will lead to the accumulation of structural information distortion during the graph representation learning process, significantly reducing the clustering performance. To this end, we propose a novel method, Structure-Missing Graph-Level Clustering Network (SMGCN), which includes a structure augmentation module LR-SEA, an Anchor Positioning Mechanism, and Joint Contrastive Optimization. Specifically, we first output augmented graphs based on low-rank matrix completion, perform cluster matching using the Hungarian algorithm to obtain anchors, and then force same clustering graphs to converge to the corresponding anchors in the embedding space. According to our research, this is the first time that the graph-level clustering task with missing relations is proposed, and the superiority of our method is demonstrated through experiments on five benchmark datasets, compared with the state-of-the-art methods. Our source codes are available at <span><span>https://github.com/MrHuSN/SMGCN</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108682"},"PeriodicalIF":6.3,"publicationDate":"2026-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146144622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-02DOI: 10.1016/j.neunet.2026.108676
Hanqi Wang , Jingyu Zhang , Peng Ye , Kun Yang , Jichuan Xiong , Xuefeng Liu , Tao Chen , Liang Song
Emotion recognition brain-computer interface (BCI) using electroencephalography (EEG) is crucial for human-computer interaction, medicine, and neuroscience. However, the scarcity of labeled EEG data limits progress in this field. To address this, self-supervised learning has gained attention as a promising approach. Despite its potential, self-supervised methods face two key challenges: (1) ensuring emotion-related information is effectively preserved, as its loss can degrade emotion recognition performance, and (2) overcoming inter-subject variability in EEG signals, which hinders generalization across subjects. To tackle these issues, we propose a novel knowledge-driven self-supervised learning framework for EEG emotion recognition. Our method incorporates domain knowledge to approximate the extraction of statistical feature differential entropy (DE), aiming to preserve emotion-related and generalizable information. The framework consists of two cascaded components as hard and soft alignments: a multi-branch convolutional differential entropy learning (MCDEL) module that simulates the DE extraction process, and a contrastive entropy alignment (CEA) module that exposes complex emotional semantics in high-dimensional space. Experiment results show that our method exhibits superior performance over existing self-supervised methods. The subject-independent mean accuracy and standard deviation of our method reached 84.48% ± 5.79 on SEED and 67.64% ± 6.35 and 68.63% ± 7.77 on the Arousal and Valence dimensions of DREAMER, respectively. We conduct an ablation study to demonstrate the contribution of each proposed component. Moreover, the t-SNE visualization intuitively presents the effect of our method on reducing inter-subject variability and discriminating emotional states.
{"title":"A knowledge-driven self-supervised learning method for enhancing EEG-based emotion recognition","authors":"Hanqi Wang , Jingyu Zhang , Peng Ye , Kun Yang , Jichuan Xiong , Xuefeng Liu , Tao Chen , Liang Song","doi":"10.1016/j.neunet.2026.108676","DOIUrl":"10.1016/j.neunet.2026.108676","url":null,"abstract":"<div><div>Emotion recognition brain-computer interface (BCI) using electroencephalography (EEG) is crucial for human-computer interaction, medicine, and neuroscience. However, the scarcity of labeled EEG data limits progress in this field. To address this, self-supervised learning has gained attention as a promising approach. Despite its potential, self-supervised methods face two key challenges: (1) ensuring emotion-related information is effectively preserved, as its loss can degrade emotion recognition performance, and (2) overcoming inter-subject variability in EEG signals, which hinders generalization across subjects. To tackle these issues, we propose a novel knowledge-driven self-supervised learning framework for EEG emotion recognition. Our method incorporates domain knowledge to approximate the extraction of statistical feature differential entropy (DE), aiming to preserve emotion-related and generalizable information. The framework consists of two cascaded components as hard and soft alignments: a multi-branch convolutional differential entropy learning (MCDEL) module that simulates the DE extraction process, and a contrastive entropy alignment (CEA) module that exposes complex emotional semantics in high-dimensional space. Experiment results show that our method exhibits superior performance over existing self-supervised methods. The subject-independent mean accuracy and standard deviation of our method reached 84.48% ± 5.79 on SEED and 67.64% ± 6.35 and 68.63% ± 7.77 on the Arousal and Valence dimensions of DREAMER, respectively. We conduct an ablation study to demonstrate the contribution of each proposed component. Moreover, the t-SNE visualization intuitively presents the effect of our method on reducing inter-subject variability and discriminating emotional states.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108676"},"PeriodicalIF":6.3,"publicationDate":"2026-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146158637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Object detection by fusing RGB frames and event streams is challenging due to their inherent heterogeneity and significant statistical disparities, which often lead to suboptimal fusion in existing methods. To address this, we introduce C3Net, a novel framework built upon a paradigm shift from direct feature merging to Collaborative Calibration. First, we propose an Adaptive Balancing Time Surface (ABTS) to generate motion-robust event representations by mitigating spatial inconsistencies caused by varying object velocities. Second, the core Cross-Modal Feature Collaborative Calibration Module (CM-FCCM) performs mutual calibration of RGB and event features across channel and spatial dimensions, reducing modality discrepancies before fusion; the calibrated features are then fed back to the respective backbones for enriched feature learning. Finally, an Adaptive Channel Fusion Module (ACFM) dynamically integrates the modalities based on channel-wise confidence. Extensive experiments on PKU-DAVIS-SOD, DSEC-MOD, and PKU-DDD17-CAR datasets demonstrate that C3Net achieves state-of-the-art performance, showcasing its superior ability to leverage the complementary strengths of frames and events.
{"title":"C3Net: A cross-modal collaborative calibration of features for object detection using frames and events","authors":"Yunhua Chen , Jinyu Zhong , Yihao Guo , Zequan Xie , Jinsheng Xiao , Pinghua Chen","doi":"10.1016/j.neunet.2026.108651","DOIUrl":"10.1016/j.neunet.2026.108651","url":null,"abstract":"<div><div>Object detection by fusing RGB frames and event streams is challenging due to their inherent heterogeneity and significant statistical disparities, which often lead to suboptimal fusion in existing methods. To address this, we introduce C3Net, a novel framework built upon a paradigm shift from direct feature merging to Collaborative Calibration. First, we propose an Adaptive Balancing Time Surface (ABTS) to generate motion-robust event representations by mitigating spatial inconsistencies caused by varying object velocities. Second, the core Cross-Modal Feature Collaborative Calibration Module (CM-FCCM) performs mutual calibration of RGB and event features across channel and spatial dimensions, reducing modality discrepancies before fusion; the calibrated features are then fed back to the respective backbones for enriched feature learning. Finally, an Adaptive Channel Fusion Module (ACFM) dynamically integrates the modalities based on channel-wise confidence. Extensive experiments on PKU-DAVIS-SOD, DSEC-MOD, and PKU-DDD17-CAR datasets demonstrate that C3Net achieves state-of-the-art performance, showcasing its superior ability to leverage the complementary strengths of frames and events.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108651"},"PeriodicalIF":6.3,"publicationDate":"2026-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-02DOI: 10.1016/j.neunet.2026.108667
Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia
Existing value-based online reinforcement learning (RL) algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Discrepancy Evolution (RDE) to facilitate Q-network representation learning, enabling discriminative representations for neighboring state-action pairs. In addition, we adopt an explicit method to policy constraints by enabling Greedy Action Guidance (GAG). This is achieved through backtracking historical actions, which effectively enhances the policy update process. Our proposed method relies on providing the learning algorithm with accurate k-nearest-neighbor action value estimates and learning to design a fast-adaptable policy through policy constraints. We further propose the Instant Policy Update (IPU) mechanism, which enhances policy exploitation by systematically increasing the frequency of policy updates. We further discover that the early-stage training conservatism of the IRA method can alleviate the overestimation bias problem in value-based RL. Experimental results show that IRA can significantly improve the learning efficiency and final performance of online RL algorithms on eight MuJoCo continuous control tasks.
{"title":"Improving policy exploitation in online reinforcement learning with instant retrospect action","authors":"Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia","doi":"10.1016/j.neunet.2026.108667","DOIUrl":"10.1016/j.neunet.2026.108667","url":null,"abstract":"<div><div>Existing value-based online reinforcement learning (RL) algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Discrepancy Evolution (RDE) to facilitate Q-network representation learning, enabling discriminative representations for neighboring state-action pairs. In addition, we adopt an explicit method to policy constraints by enabling Greedy Action Guidance (GAG). This is achieved through backtracking historical actions, which effectively enhances the policy update process. Our proposed method relies on providing the learning algorithm with accurate <em>k</em>-nearest-neighbor action value estimates and learning to design a fast-adaptable policy through policy constraints. We further propose the Instant Policy Update (IPU) mechanism, which enhances policy exploitation by systematically increasing the frequency of policy updates. We further discover that the early-stage training conservatism of the IRA method can alleviate the overestimation bias problem in value-based RL. Experimental results show that IRA can significantly improve the learning efficiency and final performance of online RL algorithms on eight MuJoCo continuous control tasks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108667"},"PeriodicalIF":6.3,"publicationDate":"2026-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146158658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-01DOI: 10.1016/j.neunet.2026.108644
Yue Zhou , Liang Cao , Yan Lei , Hongru Ren
Six-rotor unmanned aerial vehicles (UAVs) offer significant potential, but still encounter persistent challenges in achieving efficient allocation of limited resources in dynamic and complex environments. Consequently, this paper explores the prescribed-time observer-based optimal consensus control problem for six-rotor UAVs with unified prescribed performance. A practical prescribed-time optimal control scheme is constructed through embedding the prescribed-time control method with a simplified reinforcement learning framework to realize the efficient resource allocation. Leveraging a prescribed-time adjustment function, the novel updating laws for actor and critic neural networks are developed, which guarantee that six-rotor UAVs reach a desired steady state within prescribed time. Moreover, an improved distributed prescribed-time observer is established, ensuring that each follower is able to precisely estimate the velocity and position information of the leader within prescribed time. Then, a series of nonlinear transformations and mappings is proposed, which cannot only satisfy diverse performance requirements under a unified control framework through only adjusting the design parameters a priori but also improve the user-friendliness of implementation and control design. Significantly, the global performance requirement simplifies verification process of initial constraints in traditional performance control methods. Furthermore, an adaptive prescribed-time filter is introduced to address the complexity explosion issue of the backstepping method on six-rotor UAVs, while ensuring the filter error converges within prescribed time. Eventually, simulation results confirm the effectiveness of the designed method.
{"title":"Observer-based prescribed-time optimal neural consensus control for six-rotor UAVs: A novel actor-critic reinforcement learning strategy","authors":"Yue Zhou , Liang Cao , Yan Lei , Hongru Ren","doi":"10.1016/j.neunet.2026.108644","DOIUrl":"10.1016/j.neunet.2026.108644","url":null,"abstract":"<div><div>Six-rotor unmanned aerial vehicles (UAVs) offer significant potential, but still encounter persistent challenges in achieving efficient allocation of limited resources in dynamic and complex environments. Consequently, this paper explores the prescribed-time observer-based optimal consensus control problem for six-rotor UAVs with unified prescribed performance. A practical prescribed-time optimal control scheme is constructed through embedding the prescribed-time control method with a simplified reinforcement learning framework to realize the efficient resource allocation. Leveraging a prescribed-time adjustment function, the novel updating laws for actor and critic neural networks are developed, which guarantee that six-rotor UAVs reach a desired steady state within prescribed time. Moreover, an improved distributed prescribed-time observer is established, ensuring that each follower is able to precisely estimate the velocity and position information of the leader within prescribed time. Then, a series of nonlinear transformations and mappings is proposed, which cannot only satisfy diverse performance requirements under a unified control framework through only adjusting the design parameters a priori but also improve the user-friendliness of implementation and control design. Significantly, the global performance requirement simplifies verification process of initial constraints in traditional performance control methods. Furthermore, an adaptive prescribed-time filter is introduced to address the complexity explosion issue of the backstepping method on six-rotor UAVs, while ensuring the filter error converges within prescribed time. Eventually, simulation results confirm the effectiveness of the designed method.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108644"},"PeriodicalIF":6.3,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146138110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-02-01Epub Date: 2025-10-10DOI: 10.1016/j.neunet.2025.108188
Shengcai Zhang, Huiju Yi, Fanchang Zeng, Xuan Zhang, Zhiying Fu, Dezhi An
Time series forecasting is widely applied in fields such as energy and network security. Various prediction models based on Transformer and MLP architectures have been proposed. However, their performance may decline to varying degrees when applied to real-world sequences with significant non-stationarity. Traditional approaches generally adopt either stabilization or a combination of stabilization and non-stationarity compensation for prediction tasks. However, non-stationarity is a crucial attribute of time series; the former approach tends to eliminate useful non-stationary patterns, while the latter may inadequately capture non-stationary information. Therefore, we propose DiffMixer, which analyzes and predicts different frequencies in non-stationary time series. We use Variational Mode Decomposition (VMD) to obtain multiple frequency components of the sequence, Multi-scale Decomposition (MsD) to optimize the decomposition of downsampled sequences, and Improved Star Aggregate-Redistribute (iSTAR) to capture interdependencies between different frequency components. Additionally, we employ the Frequency domain Processing Block (FPB) to capture global features of different frequency components in the frequency domain, and Dual Dimension Fusion (DuDF) to fuse different frequency components in two dimensions, enhancing the predictive fit for various frequencies. Compared to previous state-of-the-art methods, DiffMixer reduces the Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Symmetric Mean Absolute Percentage Error (SMAPE) by 24.5%, 12.3%, 13.5%, and 6.1%, respectively.
{"title":"DiffMixer: A prediction model based on mixing different frequency features.","authors":"Shengcai Zhang, Huiju Yi, Fanchang Zeng, Xuan Zhang, Zhiying Fu, Dezhi An","doi":"10.1016/j.neunet.2025.108188","DOIUrl":"10.1016/j.neunet.2025.108188","url":null,"abstract":"<p><p>Time series forecasting is widely applied in fields such as energy and network security. Various prediction models based on Transformer and MLP architectures have been proposed. However, their performance may decline to varying degrees when applied to real-world sequences with significant non-stationarity. Traditional approaches generally adopt either stabilization or a combination of stabilization and non-stationarity compensation for prediction tasks. However, non-stationarity is a crucial attribute of time series; the former approach tends to eliminate useful non-stationary patterns, while the latter may inadequately capture non-stationary information. Therefore, we propose DiffMixer, which analyzes and predicts different frequencies in non-stationary time series. We use Variational Mode Decomposition (VMD) to obtain multiple frequency components of the sequence, Multi-scale Decomposition (MsD) to optimize the decomposition of downsampled sequences, and Improved Star Aggregate-Redistribute (iSTAR) to capture interdependencies between different frequency components. Additionally, we employ the Frequency domain Processing Block (FPB) to capture global features of different frequency components in the frequency domain, and Dual Dimension Fusion (DuDF) to fuse different frequency components in two dimensions, enhancing the predictive fit for various frequencies. Compared to previous state-of-the-art methods, DiffMixer reduces the Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Symmetric Mean Absolute Percentage Error (SMAPE) by 24.5%, 12.3%, 13.5%, and 6.1%, respectively.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"108188"},"PeriodicalIF":6.3,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号