Neural Networks最新文献

英文中文

A novel deep learning model combining 3DCNN-CapsNet and hierarchical attention mechanism for EEG emotion recognition

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-18 DOI: 10.1016/j.neunet.2025.107267

Kun Chen , Wenhao Ruan , Quan Liu , Qingsong Ai , Li Ma

Emotion recognition plays a key role in the field of human–computer interaction. Classifying and predicting human emotions using electroencephalogram (EEG) signals has consistently been a challenging research area. Recently, with the increasing application of deep learning methods such as convolutional neural network (CNN) and channel attention mechanism (CA). The recognition accuracy of emotion recognition methods has already reached an outstanding level. However, CNN and its derivatives have the defect that the sensory field of view is small and can only extract local features. The traditional channel attention mechanism only focuses on the correlation between different channels and assigns weights to each channel according to its contribution to the emotion recognition task, ignoring the fact that different EEG frequency bands in the same channel signal also have different contributions to the task. To address the above-mentioned problems , this paper propose HA-CapsNet, a novel end-to-end model combining 3DCNN-CapsNet with a Hierarchical Attention mechanism. This model captures both inter-channel correlations and the contribution of each frequency band. Additionally, the capsule network in 3DCNN-CapsNet extracts more spatial feature information compared to conventional CNNs. Our HA-CapsNet achieves recognition accuracies of 97.40%, 97.20%, and 97.60% on the DEAP dataset, and 95.80%, 96.10%, and 96.30% on the DREAMER dataset, outperforming state-of-the-art methods with the smallest variance. Furthermore, experiments removing channels from the DEAP and DREAMER datasets in ascending order of their hierarchical attention weights showed that even with fewer channels, the model maintained strong recognition performance. This demonstrates HA-CapsNet’s low dependence on large datasets and its suitability for lightweight EEG devices, promoting advancements in EEG device development.

{"title":"A novel deep learning model combining 3DCNN-CapsNet and hierarchical attention mechanism for EEG emotion recognition","authors":"Kun Chen , Wenhao Ruan , Quan Liu , Qingsong Ai , Li Ma","doi":"10.1016/j.neunet.2025.107267","DOIUrl":"10.1016/j.neunet.2025.107267","url":null,"abstract":"<div><div>Emotion recognition plays a key role in the field of human–computer interaction. Classifying and predicting human emotions using electroencephalogram (EEG) signals has consistently been a challenging research area. Recently, with the increasing application of deep learning methods such as convolutional neural network (CNN) and channel attention mechanism (CA). The recognition accuracy of emotion recognition methods has already reached an outstanding level. However, CNN and its derivatives have the defect that the sensory field of view is small and can only extract local features. The traditional channel attention mechanism only focuses on the correlation between different channels and assigns weights to each channel according to its contribution to the emotion recognition task, ignoring the fact that different EEG frequency bands in the same channel signal also have different contributions to the task. To address the above-mentioned problems , this paper propose HA-CapsNet, a novel end-to-end model combining 3DCNN-CapsNet with a Hierarchical Attention mechanism. This model captures both inter-channel correlations and the contribution of each frequency band. Additionally, the capsule network in 3DCNN-CapsNet extracts more spatial feature information compared to conventional CNNs. Our HA-CapsNet achieves recognition accuracies of 97.40%, 97.20%, and 97.60% on the DEAP dataset, and 95.80%, 96.10%, and 96.30% on the DREAMER dataset, outperforming state-of-the-art methods with the smallest variance. Furthermore, experiments removing channels from the DEAP and DREAMER datasets in ascending order of their hierarchical attention weights showed that even with fewer channels, the model maintained strong recognition performance. This demonstrates HA-CapsNet’s low dependence on large datasets and its suitability for lightweight EEG devices, promoting advancements in EEG device development.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107267"},"PeriodicalIF":6.0,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143479113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ZS-MNET: A zero-shot learning based approach to multimodal named entity typing

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-17 DOI: 10.1016/j.neunet.2025.107264

Baohang Zhou , Ying Zhang , Kehui Song , Xuhui Sui , Yu Zhao , Xiaojie Yuan

The task of named entity typing (NET) on social platforms is significant as it involves identifying the various types of named entities within unstructured text. The existing methods for NET only utilize the text modality to classify the types of named entities and ignore the semantic correlation of multimodal data. Moreover, the growing number of multimodal data implies a growing type set and the newly emerged entity types should be recognized without additional training. To address the aforementioned disadvantages, we introduce a zero-shot learning based multimodal NET (ZS-MNET) model that combines textual and visual modalities to recognize previously unseen named entity types in a zero-shot manner. The proposed ZS-MNET utilizes both text and image information to bridge the semantic correlation between multimodal data and label information, as opposed to the traditional zero-shot NET (ZS-NET) models. To incorporate fine-grained multimodal representations, we utilize pre-trained models that incorporate language and vision, particularly BERT and ViT, which are founded on transformer architectures. Besides, we propose the different multimodal representations to focus on fine-grained features for modeling semantic correlation between multimodal data and entity types in a fusion way. The experimental results underscore the utility of multimodal data in the NET field, while our approach surpasses previous ZS-NET models in performance.

{"title":"ZS-MNET: A zero-shot learning based approach to multimodal named entity typing","authors":"Baohang Zhou , Ying Zhang , Kehui Song , Xuhui Sui , Yu Zhao , Xiaojie Yuan","doi":"10.1016/j.neunet.2025.107264","DOIUrl":"10.1016/j.neunet.2025.107264","url":null,"abstract":"<div><div>The task of named entity typing (NET) on social platforms is significant as it involves identifying the various types of named entities within unstructured text. The existing methods for NET only utilize the text modality to classify the types of named entities and ignore the semantic correlation of multimodal data. Moreover, the growing number of multimodal data implies a growing type set and the newly emerged entity types should be recognized without additional training. To address the aforementioned disadvantages, we introduce a zero-shot learning based multimodal NET (ZS-MNET) model that combines textual and visual modalities to recognize previously unseen named entity types in a zero-shot manner. The proposed ZS-MNET utilizes both text and image information to bridge the semantic correlation between multimodal data and label information, as opposed to the traditional zero-shot NET (ZS-NET) models. To incorporate fine-grained multimodal representations, we utilize pre-trained models that incorporate language and vision, particularly BERT and ViT, which are founded on transformer architectures. Besides, we propose the different multimodal representations to focus on fine-grained features for modeling semantic correlation between multimodal data and entity types in a fusion way. The experimental results underscore the utility of multimodal data in the NET field, while our approach surpasses previous ZS-NET models in performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107264"},"PeriodicalIF":6.0,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143487520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Neural transition system abstraction for neural network dynamical system models and its application to Computational Tree Logic verification

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-15 DOI: 10.1016/j.neunet.2025.107261

Yejiang Yang , Tao Wang , Weiming Xiang

This paper proposes an explainable abstraction-based verification method that prioritizes user interaction and enhances interpretability. By partitioning the system’s state space using a data-driven process, we can abstract the dynamics into words consisting of state labels. When given a trained neural network model, a set-valued reachability analysis method is introduced to estimate the relationship between each subsystem. We construct the neural transition system abstraction with the neural network model and the relationships between partitions. Then, the abstracted model can be verified through Computational Tree Logic (CTL), enabling formal verification of the system’s behavior. This approach greatly enhances the interpretability and verification of data-driven models, as well as the ability to validate against the specification. Finally, examples of the Maglev model and handwritten model abstractions are given to illustrate our proposed model verification framework, which demonstrates that the proposed framework has advantages in enhancing model interpretability and verifying user-specified properties based on CTL.

{"title":"Neural transition system abstraction for neural network dynamical system models and its application to Computational Tree Logic verification","authors":"Yejiang Yang , Tao Wang , Weiming Xiang","doi":"10.1016/j.neunet.2025.107261","DOIUrl":"10.1016/j.neunet.2025.107261","url":null,"abstract":"<div><div>This paper proposes an explainable abstraction-based verification method that prioritizes user interaction and enhances interpretability. By partitioning the system’s state space using a data-driven process, we can abstract the dynamics into words consisting of state labels. When given a trained neural network model, a set-valued reachability analysis method is introduced to estimate the relationship between each subsystem. We construct the neural transition system abstraction with the neural network model and the relationships between partitions. Then, the abstracted model can be verified through Computational Tree Logic (CTL), enabling formal verification of the system’s behavior. This approach greatly enhances the interpretability and verification of data-driven models, as well as the ability to validate against the specification. Finally, examples of the Maglev model and handwritten model abstractions are given to illustrate our proposed model verification framework, which demonstrates that the proposed framework has advantages in enhancing model interpretability and verifying user-specified properties based on CTL.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107261"},"PeriodicalIF":6.0,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143474197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hybrid multi-modality multi-task learning for forecasting progression trajectories in subjective cognitive decline

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-15 DOI: 10.1016/j.neunet.2025.107263

Minhui Yu , Yuqi Fang , Yunbi Liu , Andrea C. Bozoki , Shifu Xiao , Ling Yue , Mingxia Liu

While numerous studies strive to exploit the complementary potential of MRI and PET using learning-based methods, the effective fusion of the two modalities remains a tricky problem due to their inherently distinctive properties. In addition, current studies often face the problem of small sample sizes and missing PET data due to factors such as patient withdrawal or low image quality. To this end, we propose a hybrid multi-modality multi-task learning (HM

^{2}

L) framework with cross-domain knowledge transfer for forecasting trajectories of SCD progression. Our HM

^{2}

L comprises (1) missing PET imputation, (2) multi-modality feature extraction for MRI and PET feature learning with a novel softmax-triplet constraint, (3) attention-based multi-modality fusion of MRI and PET features, and (4) multi-task prediction of category labels and clinical scores such as Mini-Mental State Examination (MMSE) and Geriatric Depression Scale (GDS). To handle problems with small sample sizes, a transfer learning strategy is developed to enable knowledge transfer from a relatively large scale dataset with MRI and PET from 795 subjects to two small-scale SCD cohorts with a total of 136 subjects. Experimental results indicate HM

^{2}

L surpasses several state-of-the-art methods in jointly predicting category labels and clinical scores of subjective cognitive decline. Results show that the MMSE scores of SCD subjects who develop mild cognitive impairment during the 2-year/7-year follow-up are significantly lower than those of subjects who remain stable, while there exists a complex relationship between SCD progression with GDS.

{"title":"Hybrid multi-modality multi-task learning for forecasting progression trajectories in subjective cognitive decline","authors":"Minhui Yu , Yuqi Fang , Yunbi Liu , Andrea C. Bozoki , Shifu Xiao , Ling Yue , Mingxia Liu","doi":"10.1016/j.neunet.2025.107263","DOIUrl":"10.1016/j.neunet.2025.107263","url":null,"abstract":"<div><div>While numerous studies strive to exploit the complementary potential of MRI and PET using learning-based methods, the effective fusion of the two modalities remains a tricky problem due to their inherently distinctive properties. In addition, current studies often face the problem of small sample sizes and missing PET data due to factors such as patient withdrawal or low image quality. To this end, we propose a hybrid multi-modality multi-task learning (HM<span><math><msup><mrow></mrow><mrow><mn>2</mn></mrow></msup></math></span>L) framework with cross-domain knowledge transfer for forecasting trajectories of SCD progression. Our HM<span><math><msup><mrow></mrow><mrow><mn>2</mn></mrow></msup></math></span>L comprises (1) <em>missing PET imputation</em>, (2) <em>multi-modality feature extraction</em> for MRI and PET feature learning with a novel softmax-triplet constraint, (3) attention-based <em>multi-modality fusion</em> of MRI and PET features, and (4) <em>multi-task prediction</em> of category labels and clinical scores such as Mini-Mental State Examination (MMSE) and Geriatric Depression Scale (GDS). To handle problems with small sample sizes, a transfer learning strategy is developed to enable knowledge transfer from a relatively large scale dataset with MRI and PET from 795 subjects to two small-scale SCD cohorts with a total of 136 subjects. Experimental results indicate HM<span><math><msup><mrow></mrow><mrow><mn>2</mn></mrow></msup></math></span>L surpasses several state-of-the-art methods in jointly predicting category labels and clinical scores of subjective cognitive decline. Results show that the MMSE scores of SCD subjects who develop mild cognitive impairment during the 2-year/7-year follow-up are significantly lower than those of subjects who remain stable, while there exists a complex relationship between SCD progression with GDS.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107263"},"PeriodicalIF":6.0,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143453693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning temporal regularized spatial-aware deep correlation filter tracking via adaptive channel selection

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-15 DOI: 10.1016/j.neunet.2025.107210

Sathiyamoorthi Arthanari, Dinesh Elayaperumal, Young Hoon Joo

In recent years, deep correlation filters have demonstrated outstanding performance in robust object tracking. Nevertheless, the correlation filters encounter challenges in managing huge occlusion, target deviation, and background clutter due to the lack of effective utilization of previous target information. To overcome these issues, we propose a novel temporal regularized spatial-aware deep correlation filter tracking via adaptive channel selection. To do this, we first presented the adaptive channel selection approach, which efficiently handles target deviation by adaptively selecting suitable channels during the learning stage. In addition, the adaptive channel selection method allows for dynamic adjustments to the filter based on the unique characteristics of the target object. This adaptability enhances the tracker’s flexibility, making it well-suited for diverse tracking scenarios. Second, we propose the spatial-aware correlation filter with dynamic spatial constraints, which effectively reduces the filter response in the complex background region by distinguishing between the foreground and background regions in the response map. Hence, the target can be easily identified within the foreground region. Third, we designed a temporal regularization approach that improves the target accuracy when the case of large appearance variations. Additionally, this temporal regularization method considers the present and previous frames of the target region, which significantly enhances the tracking ability by utilizing historical information. Finally, we present a comprehensive experiments analysis of the OTB-2013, OTB-2015, TempleColor-128, UAV-123, UAVDT, and DTB-70 benchmark datasets to demonstrate the effectiveness of the proposed approach against the state-of-the-trackers.

{"title":"Learning temporal regularized spatial-aware deep correlation filter tracking via adaptive channel selection","authors":"Sathiyamoorthi Arthanari, Dinesh Elayaperumal, Young Hoon Joo","doi":"10.1016/j.neunet.2025.107210","DOIUrl":"10.1016/j.neunet.2025.107210","url":null,"abstract":"<div><div>In recent years, deep correlation filters have demonstrated outstanding performance in robust object tracking. Nevertheless, the correlation filters encounter challenges in managing huge occlusion, target deviation, and background clutter due to the lack of effective utilization of previous target information. To overcome these issues, we propose a novel temporal regularized spatial-aware deep correlation filter tracking via adaptive channel selection. To do this, we first presented the adaptive channel selection approach, which efficiently handles target deviation by adaptively selecting suitable channels during the learning stage. In addition, the adaptive channel selection method allows for dynamic adjustments to the filter based on the unique characteristics of the target object. This adaptability enhances the tracker’s flexibility, making it well-suited for diverse tracking scenarios. Second, we propose the spatial-aware correlation filter with dynamic spatial constraints, which effectively reduces the filter response in the complex background region by distinguishing between the foreground and background regions in the response map. Hence, the target can be easily identified within the foreground region. Third, we designed a temporal regularization approach that improves the target accuracy when the case of large appearance variations. Additionally, this temporal regularization method considers the present and previous frames of the target region, which significantly enhances the tracking ability by utilizing historical information. Finally, we present a comprehensive experiments analysis of the OTB-2013, OTB-2015, TempleColor-128, UAV-123, UAVDT, and DTB-70 benchmark datasets to demonstrate the effectiveness of the proposed approach against the state-of-the-trackers.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107210"},"PeriodicalIF":6.0,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143463656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Distributed multi-timescale algorithm for nonconvex optimization problem: A control perspective

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-15 DOI: 10.1016/j.neunet.2025.107257

Xiasheng Shi , Jian Liu , Changyin Sun

The distributed nonconvex constrained optimization problem with equality and inequality constraints is researched in this paper, where the objective function and the function for constraints are all nonconvex. To solve this problem from a control perspective, a virtual reference-based convex penalty function is added to the augmented Lagrangian function. Then, based on the primal–dual technique, a two-timescale distributed approach is designed based on the consensus scheme. The slower subsystem aims to ensure the optimality, and the faster subsystem intends to guarantee the stability. Finally, three cases are presented to illustrate the approach’s effectiveness.

引用次数: 0

Don’t fear peculiar activation functions: EUAF and beyond

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-14 DOI: 10.1016/j.neunet.2025.107258

Qianchao Wang , Shijun Zhang , Dong Zeng , Zhaoheng Xie , Hengtao Guo , Tieyong Zeng , Feng-Lei Fan

In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEUAF through systematic and comprehensive experiments on various industrial and image datasets, including CIFAR-10, Tiny-ImageNet, and ImageNet. The models utilizing PEUAF achieve the best performance across several baseline industrial datasets. Specifically, in image datasets, the models that incorporate mixed activation functions (with PEUAF) exhibit competitive test accuracy despite the low accuracy of models with only PEUAF. Moreover, we significantly generalize the family of super-expressive activation functions, whose existence has been demonstrated in several recent works by showing that any continuous function can be approximated to any desired accuracy by a fixed-size network with a specific super-expressive activation function. Specifically, our work addresses two major bottlenecks in impeding the development of super-expressive activation functions: the limited identification of super-expressive functions, which raises doubts about their broad applicability, and their often peculiar forms, which lead to skepticism regarding their scalability and practicality in real-world applications.

{"title":"Don’t fear peculiar activation functions: EUAF and beyond","authors":"Qianchao Wang , Shijun Zhang , Dong Zeng , Zhaoheng Xie , Hengtao Guo , Tieyong Zeng , Feng-Lei Fan","doi":"10.1016/j.neunet.2025.107258","DOIUrl":"10.1016/j.neunet.2025.107258","url":null,"abstract":"<div><div>In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEUAF through systematic and comprehensive experiments on various industrial and image datasets, including CIFAR-10, Tiny-ImageNet, and ImageNet. The models utilizing PEUAF achieve the best performance across several baseline industrial datasets. Specifically, in image datasets, the models that incorporate mixed activation functions (with PEUAF) exhibit competitive test accuracy despite the low accuracy of models with only PEUAF. Moreover, we significantly generalize the family of super-expressive activation functions, whose existence has been demonstrated in several recent works by showing that any continuous function can be approximated to any desired accuracy by a fixed-size network with a specific super-expressive activation function. Specifically, our work addresses two major bottlenecks in impeding the development of super-expressive activation functions: the limited identification of super-expressive functions, which raises doubts about their broad applicability, and their often peculiar forms, which lead to skepticism regarding their scalability and practicality in real-world applications.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107258"},"PeriodicalIF":6.0,"publicationDate":"2025-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143471607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spiking neural networks on FPGA: A survey of methodologies and recent advancements

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-14 DOI: 10.1016/j.neunet.2025.107256

Mehrzad Karamimanesh , Ebrahim Abiri , Mahyar Shahsavari , Kourosh Hassanli , André van Schaik , Jason Eshraghian

The mimicry of the biological brain’s structure in information processing enables spiking neural networks (SNNs) to exhibit significantly reduced power consumption compared to conventional systems. Consequently, these networks have garnered heightened attention and spurred extensive research endeavors in recent years, proposing various structures to achieve low power consumption, high speed, and improved recognition ability. However, researchers are still in the early stages of developing more efficient neural networks that more closely resemble the biological brain. This development and research require suitable hardware for execution with appropriate capabilities, and field-programmable gate array (FPGA) serves as a highly qualified candidate compared to existing hardware such as central processing unit (CPU) and graphics processing unit (GPU). FPGA, with parallel processing capabilities similar to the brain, lower latency and power consumption, and higher throughput, is highly eligible hardware for assisting in the development of spiking neural networks. In this review, an attempt has been made to facilitate researchers’ path to further develop this field by collecting and examining recent works and the challenges that hinder the implementation of these networks on FPGA.

{"title":"Spiking neural networks on FPGA: A survey of methodologies and recent advancements","authors":"Mehrzad Karamimanesh , Ebrahim Abiri , Mahyar Shahsavari , Kourosh Hassanli , André van Schaik , Jason Eshraghian","doi":"10.1016/j.neunet.2025.107256","DOIUrl":"10.1016/j.neunet.2025.107256","url":null,"abstract":"<div><div>The mimicry of the biological brain’s structure in information processing enables spiking neural networks (SNNs) to exhibit significantly reduced power consumption compared to conventional systems. Consequently, these networks have garnered heightened attention and spurred extensive research endeavors in recent years, proposing various structures to achieve low power consumption, high speed, and improved recognition ability. However, researchers are still in the early stages of developing more efficient neural networks that more closely resemble the biological brain. This development and research require suitable hardware for execution with appropriate capabilities, and field-programmable gate array (FPGA) serves as a highly qualified candidate compared to existing hardware such as central processing unit (CPU) and graphics processing unit (GPU). FPGA, with parallel processing capabilities similar to the brain, lower latency and power consumption, and higher throughput, is highly eligible hardware for assisting in the development of spiking neural networks. In this review, an attempt has been made to facilitate researchers’ path to further develop this field by collecting and examining recent works and the challenges that hinder the implementation of these networks on FPGA.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107256"},"PeriodicalIF":6.0,"publicationDate":"2025-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143422441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dual view graph transformer networks for multi-hop knowledge graph reasoning

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-14 DOI: 10.1016/j.neunet.2025.107260

Congcong Sun , Jianrui Chen , Zhongshi Shao , Junjie Huang

To address the incompleteness of knowledge graphs, multi-hop reasoning aims to find the unknown information from existing data and enhance the comprehensive understanding. The presence of reasoning paths endows multi-hop reasoning with interpretability and traceability. Existing reinforcement learning (RL)-based multi-hop reasoning methods primarily rely on the agent’s blind trial-and-error approach in a large search space, which leads to inefficient training. In contrast, sequence-based multi-hop reasoning methods focus on learning the mapping from path to path to achieve better training efficiency, but they discard structured knowledge. The absence of structured knowledge directly hinders the ablity to capture and represent complex relations. To address the above issues, we propose a Dual View Graph Transformer Networks for Multi-hop Knowledge Graph Reasoning (DV4KGR), which enables the joint learning of structured and serialized views. The structured view contains a large amount of structured knowledge, which represents the relations among nodes from a global perspective. Meanwhile, the serialized view contains rich knowledge of reasoning semantics, aiding in training the mapping function from reasoning states to reasoning paths. We learn the representations of one-to-many relations in a supervised contrastive learning manner, which enhances the ability to represent complex relations. Additionally, we combine structured knowledge and rule induction for action smoothing, which effectively alleviates the overfitting problem associated with the end-to-end training mode. The experimental results on four benchmark datasets demonstrate that DV4KGR delivers better performance than the state-of-the-art baselines.

{"title":"Dual view graph transformer networks for multi-hop knowledge graph reasoning","authors":"Congcong Sun , Jianrui Chen , Zhongshi Shao , Junjie Huang","doi":"10.1016/j.neunet.2025.107260","DOIUrl":"10.1016/j.neunet.2025.107260","url":null,"abstract":"<div><div>To address the incompleteness of knowledge graphs, multi-hop reasoning aims to find the unknown information from existing data and enhance the comprehensive understanding. The presence of reasoning paths endows multi-hop reasoning with interpretability and traceability. Existing reinforcement learning (RL)-based multi-hop reasoning methods primarily rely on the agent’s blind trial-and-error approach in a large search space, which leads to inefficient training. In contrast, sequence-based multi-hop reasoning methods focus on learning the mapping from path to path to achieve better training efficiency, but they discard structured knowledge. The absence of structured knowledge directly hinders the ablity to capture and represent complex relations. To address the above issues, we propose a <strong>D</strong>ual <strong>V</strong>iew Graph Transformer Networks <strong>f</strong>or Multi-hop <strong>K</strong>nowledge <strong>G</strong>raph <strong>R</strong>easoning (DV4KGR), which enables the joint learning of structured and serialized views. The structured view contains a large amount of structured knowledge, which represents the relations among nodes from a global perspective. Meanwhile, the serialized view contains rich knowledge of reasoning semantics, aiding in training the mapping function from reasoning states to reasoning paths. We learn the representations of one-to-many relations in a supervised contrastive learning manner, which enhances the ability to represent complex relations. Additionally, we combine structured knowledge and rule induction for action smoothing, which effectively alleviates the overfitting problem associated with the end-to-end training mode. The experimental results on four benchmark datasets demonstrate that DV4KGR delivers better performance than the state-of-the-art baselines.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107260"},"PeriodicalIF":6.0,"publicationDate":"2025-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143429249","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Heterogeneous boundary synchronization of time-delayed competitive neural networks with adaptive learning parameter in the space-time discretized frames

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neural Networks

Pub Date : 2025-02-13 DOI: 10.1016/j.neunet.2025.107255

Tianwei Zhang , Shaobin Rao , Jianwen Zhou

This article presents the master-slave time-delayed competitive neural networks in space-time discretized frames (STD-CNNs) with the heterogeneous structure, induced by the design of an adaptive learning parameter in the slave STD-CNNs. This article addresses the issue of exponential synchronization for the time-delayed STD-CNNs with the heterogeneous structure via the controls at the boundaries, based on the learning law setting for the parameter in the slave STD-CNNs. In a corresponding manner, the exponential synchronization for time-delayed STD-CNNs with the homogeneous structure can be achieved via boundary controls. This study demonstrates that the problem of exponential synchronization for time-delayed heterogeneous STD-CNNs can be modeled by designating a time-varying learning parameter in the slave STD-CNNs, which can then be solved by means of calculative linear matrix inequalities (LMIs). To illustrate the feasibility of the current work, a numerical example is presented.

引用次数: 0

首页上一页

下一页尾页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Neural Networks

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀