Neurocomputing最新文献_第5页

Data-specific activation function learning for constructive neural networks 建设性神经网络的数据特定激活函数学习

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.129020

Zhenxing Xia , Wei Dai , Xin Liu , Haijun Zhang , Xiaoping Ma

Activation functions play a crucial role in learning and expressive capabilities of advanced neural networks due to their non-linear or non-saturated properties. However, how to determine the appropriate activation function from various candidates is a challenging yet not well-addressed topic. To address the issue, a novel self-learning approach, called as data-specific activation function learning (DSAFL) algorithm, is proposed to establish constructive neural network on one-time by adaptively selecting appropriate activation function based on the specific data characteristics. To assess the space dimension mapping abilities of different activation functions, the configuration probabilities are used to guide the generation of various candidate activation functions and corresponding candidate hidden node. In the learning stage, an exploration-exploitation mechanism composed of the random algorithm and the greedy strategy is developed to obtain the influence of different candidate activation functions, thereby avoiding configuration probabilities falling into local optimum. A reward-penalty mechanism is built to update the configuration probabilities and enhance the robustness of network by integrating the simulated annealing strategy. In final, the activation function with the highest configuration probability, as the best one, is used to reconstruct the neural network. Experimental results on both regression and classification tasks demonstrate the efficiency and effectiveness of DSAFL in the activation function selection problems of a class of constructive neural networks.

激活函数由于其非线性或非饱和的特性，在高级神经网络的学习和表达能力中起着至关重要的作用。然而，如何从各种候选中确定合适的激活函数是一个具有挑战性但尚未得到很好解决的话题。为了解决这一问题，提出了一种新的自学习方法——数据特定激活函数学习（data-specific activation function learning， DSAFL）算法，该算法根据特定的数据特征自适应选择合适的激活函数，一次性建立建设性神经网络。为了评估不同激活函数的空间维度映射能力，利用构型概率指导生成各种候选激活函数和相应的候选隐藏节点。在学习阶段，提出了一种由随机算法和贪婪策略组成的探索-开发机制，以获取不同候选激活函数的影响，从而避免构型概率陷入局部最优。结合模拟退火策略，建立奖惩机制，更新配置概率，增强网络的鲁棒性。最后，选取配置概率最高的激活函数作为最佳激活函数，对神经网络进行重构。在回归和分类任务上的实验结果都证明了DSAFL在一类构造性神经网络的激活函数选择问题中的效率和有效性。

{"title":"Data-specific activation function learning for constructive neural networks","authors":"Zhenxing Xia , Wei Dai , Xin Liu , Haijun Zhang , Xiaoping Ma","doi":"10.1016/j.neucom.2024.129020","DOIUrl":"10.1016/j.neucom.2024.129020","url":null,"abstract":"<div><div>Activation functions play a crucial role in learning and expressive capabilities of advanced neural networks due to their non-linear or non-saturated properties. However, how to determine the appropriate activation function from various candidates is a challenging yet not well-addressed topic. To address the issue, a novel self-learning approach, called as data-specific activation function learning (DSAFL) algorithm, is proposed to establish constructive neural network on one-time by adaptively selecting appropriate activation function based on the specific data characteristics. To assess the space dimension mapping abilities of different activation functions, the configuration probabilities are used to guide the generation of various candidate activation functions and corresponding candidate hidden node. In the learning stage, an exploration-exploitation mechanism composed of the random algorithm and the greedy strategy is developed to obtain the influence of different candidate activation functions, thereby avoiding configuration probabilities falling into local optimum. A reward-penalty mechanism is built to update the configuration probabilities and enhance the robustness of network by integrating the simulated annealing strategy. In final, the activation function with the highest configuration probability, as the best one, is used to reconstruct the neural network. Experimental results on both regression and classification tasks demonstrate the efficiency and effectiveness of DSAFL in the activation function selection problems of a class of constructive neural networks.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"617 ","pages":"Article 129020"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142759366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Gossip-based asynchronous algorithms for distributed composite optimization 基于gossip的分布式复合优化异步算法

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128952

Xianju Fang, Baoyong Zhang, Deming Yuan

The distributed composite optimization problem associated a multi-agent network is investigated in this paper. Different from conventional optimization issues, the cost function of composite optimization consists of a convex function and a regularization function (possibly nonsmooth). The gossip protocol is also introduced to enhance the robustness of the network, and a gossip-based distributed composite mirror descent algorithm is presented to deal with the previous problem, which adopts the asynchronous communication method. Moreover, the algorithm performance is analyzed and the theoretical results on the corresponding error bounds are obtained. Finally, the distributed logistic regression is provided as an example to validate the practicability of the proposed algorithm.

研究了与多智能体网络相关的分布式复合优化问题。与传统的优化问题不同，复合优化的代价函数由一个凸函数和一个正则化函数（可能是非光滑的）组成。为了提高网络的鲁棒性，引入了八卦协议，并提出了一种基于八卦的分布式复合镜像下降算法来解决上述问题，该算法采用异步通信方式。并对算法进行了性能分析，得到了相应误差界的理论结果。最后，以分布式逻辑回归为例验证了该算法的实用性。

引用次数: 0

EMPViT: Efficient multi-path vision transformer for security risks detection in power distribution network 用于配电网安全风险检测的高效多径视觉变压器

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128967

Pan Li, Xiaofang Yuan, Haozhi Xu, Jinlei Wang, Yaonan Wang

To maintain the safe operation of power distribution network (PDN) equipment, it is important to accurately and promptly identify security risks. However, conventional drone-based object detection methods face challenges due to noise and similarity features in risk targets, as well as limited computing resources of unmanned aerial vehicles (UAVs). To address these challenges, an efficient embedding-based multi-path fusion architecture is proposed. This architecture uses a re-parameterized depthwise block to embed local context information at different scales, enhancing the extraction of tiny features while preserving inference speed. Additionally, a coordinated self-attention module is proposed to reduce computational complexity while maintaining the performance of global information. By fusing fine and coarse feature representations without requiring a lot of computation, this module efficiently learns from both local and global features from images. The goal is to create an efficient multi-path vision transformer (EMPViT) architecture that achieves a balance between accuracy and efficiency. The proposed EMPViT has been evaluated on two different drone image dataset, demonstrating better performance compared to other architectures. Specifically, the EMPViT-S improves the detection mAP by 1.2%, and the inference speed is improved to 1.24 times on average on Drone-PDN dataset. It has achieved the same performance improvement on VisDrone-DET2019 dataset, gaining detection performance by 1.3% and 1.2 times acceleration on average.

为了维护PDN设备的安全运行，准确、及时地识别安全隐患是十分重要的。然而，传统的基于无人机的目标检测方法由于风险目标的噪声和相似性特征以及无人机计算资源有限而面临挑战。为了解决这些问题，提出了一种高效的基于嵌入的多路径融合体系结构。该体系结构使用重新参数化的深度块来嵌入不同尺度的局部上下文信息，在保持推理速度的同时增强了对微小特征的提取。此外，为了在保持全局信息性能的同时降低计算复杂度，提出了一种协调的自关注模块。通过在不需要大量计算的情况下融合精细和粗糙的特征表示，该模块可以有效地从图像中学习局部和全局特征。目标是创建一个高效的多路径视觉转换器（EMPViT）体系结构，在准确性和效率之间取得平衡。在两个不同的无人机图像数据集上对所提出的EMPViT进行了评估，与其他架构相比，显示出更好的性能。其中，EMPViT-S在无人机- pdn数据集上的检测mAP提高了1.2%，推理速度平均提高到1.24倍。它在VisDrone-DET2019数据集上取得了相同的性能提升，检测性能平均提高1.3%，加速度平均提高1.2倍。

{"title":"EMPViT: Efficient multi-path vision transformer for security risks detection in power distribution network","authors":"Pan Li, Xiaofang Yuan, Haozhi Xu, Jinlei Wang, Yaonan Wang","doi":"10.1016/j.neucom.2024.128967","DOIUrl":"10.1016/j.neucom.2024.128967","url":null,"abstract":"<div><div>To maintain the safe operation of power distribution network (PDN) equipment, it is important to accurately and promptly identify security risks. However, conventional drone-based object detection methods face challenges due to noise and similarity features in risk targets, as well as limited computing resources of unmanned aerial vehicles (UAVs). To address these challenges, an efficient embedding-based multi-path fusion architecture is proposed. This architecture uses a re-parameterized depthwise block to embed local context information at different scales, enhancing the extraction of tiny features while preserving inference speed. Additionally, a coordinated self-attention module is proposed to reduce computational complexity while maintaining the performance of global information. By fusing fine and coarse feature representations without requiring a lot of computation, this module efficiently learns from both local and global features from images. The goal is to create an efficient multi-path vision transformer (EMPViT) architecture that achieves a balance between accuracy and efficiency. The proposed EMPViT has been evaluated on two different drone image dataset, demonstrating better performance compared to other architectures. Specifically, the EMPViT-S improves the detection mAP by 1.2%, and the inference speed is improved to 1.24 times on average on Drone-PDN dataset. It has achieved the same performance improvement on VisDrone-DET2019 dataset, gaining detection performance by 1.3% and 1.2 times acceleration on average.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"617 ","pages":"Article 128967"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142745303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fixed-time Lyapunov criteria of stochastic impulsive time-delay systems and its application to synchronization of Chua’s circuit networks 随机脉冲时滞系统的定时Lyapunov准则及其在Chua电路网络同步中的应用

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128943

Xiaofei Xing

In this paper, we investigate the fixed-time stability and synchronization issues of stochastic impulsive delay complex networks (SICNs) with delayed impulses. Firstly, a new fixed-time stability criterion, which provides a direct connection between impulsive strength, impulsive instants, delays and the stability time, is established for stochastic delay systems with delayed impulsive. In this criterion, two types of delays: system delay and impulsive delay are considered, and the impulsive intensity at each impulse instant can be different, (facilitate or disrupt stability). It is the first time to consider the fixed-time stability issues of this case. Secondly, based on this criterion, a novel feedback controller and an adaptive controller are designed to study the fixed-time synchronization of SICNs with delayed impulses; Thirdly, by utilizing Lyapunov functional theory, stochastic analysis technique, matrix inequality analysis techniques and proposed stability criterion, the fixed-time synchronization conditions of SICNs, are derived in the form of linear matrix inequalities (LMIs). Finally, a numerical example and an application example, Chua’s circuit networks, are provided to illustrate the validity of the theoretical analysis.

研究了具有延迟脉冲的随机脉冲延迟复杂网络（sicn）的定时稳定性和同步问题。首先，针对具有延迟脉冲的随机时滞系统，建立了一个新的定时稳定性判据，该判据提供了脉冲强度、脉冲瞬间、时滞和稳定时间之间的直接联系；在该判据中，考虑了两种类型的延迟：系统延迟和脉冲延迟，并且每个脉冲时刻的脉冲强度可以不同，（有利于或破坏稳定）。这是第一次考虑这种情况下的固定时间稳定性问题。其次，基于该准则，设计了一种新的反馈控制器和自适应控制器，研究了具有延迟脉冲的sicn的定时同步问题；第三，利用Lyapunov泛函理论、随机分析技术、矩阵不等式分析技术和提出的稳定性判据，以线性矩阵不等式（lmi）的形式导出了sicn的定时同步条件。最后，给出了一个数值算例和蔡氏电路网络的应用实例来说明理论分析的有效性。

{"title":"Fixed-time Lyapunov criteria of stochastic impulsive time-delay systems and its application to synchronization of Chua’s circuit networks","authors":"Xiaofei Xing","doi":"10.1016/j.neucom.2024.128943","DOIUrl":"10.1016/j.neucom.2024.128943","url":null,"abstract":"<div><div>In this paper, we investigate the fixed-time stability and synchronization issues of stochastic impulsive delay complex networks (SICNs) with delayed impulses. Firstly, a new fixed-time stability criterion, which provides a direct connection between impulsive strength, impulsive instants, delays and the stability time, is established for stochastic delay systems with delayed impulsive. In this criterion, two types of delays: system delay and impulsive delay are considered, and the impulsive intensity at each impulse instant can be different, (facilitate or disrupt stability). It is the first time to consider the fixed-time stability issues of this case. Secondly, based on this criterion, a novel feedback controller and an adaptive controller are designed to study the fixed-time synchronization of SICNs with delayed impulses; Thirdly, by utilizing Lyapunov functional theory, stochastic analysis technique, matrix inequality analysis techniques and proposed stability criterion, the fixed-time synchronization conditions of SICNs, are derived in the form of linear matrix inequalities (LMIs). Finally, a numerical example and an application example, Chua’s circuit networks, are provided to illustrate the validity of the theoretical analysis.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128943"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ICFNet: Interactive-complementary fusion network for monocular 3D human pose estimation ICFNet：用于单目三维人体姿态估计的交互互补融合网络

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128947

Yong Wang , Peng Liu , Hongbo Kang , Doudou Wu , Duoqian Miao

Most existing methods for 3D human pose estimation from monocular images focus on learning the spatial correlation of either the global or local joints of the human body but fail to adequately capture the inherent dependencies between them. To address this limitation, we propose the Interactive Complementary Fusion Network (ICFNet), an algorithm designed to fully utilize the prior knowledge of both global and local joint relationships to enhance prediction performance. Specifically, we introduce two feature capturers: the Global Knowledge Prior Capturer (GKPC) and the Local Region Subject Capturer (LRSC), which respectively capture global body knowledge and local joint information. Additionally, we propose three joint constraint mechanisms to express the potential association dependencies between global and local joints, which are further modeled using two association capturers: the Refined-Regression Association Capture Module (RR-ACM) and the Generalized-Guidance Association Capture Module (GG-ACM). Moreover, we introduce a novel feature transformation module, the Link Conversion Module (LCM), to transform and augment pose features. The algorithm adopts a complementary process to enhance the interaction and fusion of global and local feature information by gradually imposing constraints on the physical topological features of the human body, thereby improving its modeling capabilities. Extensive experiments demonstrate that our proposed ICFNet achieves state-of-the-art results on two challenging benchmark datasets: Human 3.6M and MPI-INF-3DHP. The code and model are available at: https://github.com/PENG-LAU/ICFNet.

现有的基于单眼图像的三维人体姿态估计方法大多侧重于学习人体整体或局部关节的空间相关性，但未能充分捕捉到它们之间的内在依赖关系。为了解决这一限制，我们提出了交互式互补融合网络（ICFNet），该算法旨在充分利用全局和局部联合关系的先验知识来提高预测性能。具体来说，我们引入了两个特征捕获器：全局知识先验捕获器（GKPC）和局部区域主题捕获器（LRSC），它们分别捕获全局主体知识和局部联合信息。此外，我们提出了三种联合约束机制来表达全局和局部关节之间潜在的关联依赖关系，并使用两个关联捕获器进一步建模：精细回归关联捕获模块（RR-ACM）和广义制导关联捕获模块（GG-ACM）。此外，我们还引入了一种新的特征转换模块——链路转换模块（Link Conversion module， LCM），用于变换和增强姿态特征。该算法采用互补过程，通过逐步对人体物理拓扑特征施加约束，增强全局与局部特征信息的交互与融合，从而提高建模能力。大量的实验表明，我们提出的ICFNet在两个具有挑战性的基准数据集（Human 3.6M和MPI-INF-3DHP）上取得了最先进的结果。代码和模型可在https://github.com/PENG-LAU/ICFNet上获得。

{"title":"ICFNet: Interactive-complementary fusion network for monocular 3D human pose estimation","authors":"Yong Wang , Peng Liu , Hongbo Kang , Doudou Wu , Duoqian Miao","doi":"10.1016/j.neucom.2024.128947","DOIUrl":"10.1016/j.neucom.2024.128947","url":null,"abstract":"<div><div>Most existing methods for 3D human pose estimation from monocular images focus on learning the spatial correlation of either the global or local joints of the human body but fail to adequately capture the inherent dependencies between them. To address this limitation, we propose the Interactive Complementary Fusion Network (ICFNet), an algorithm designed to fully utilize the prior knowledge of both global and local joint relationships to enhance prediction performance. Specifically, we introduce two feature capturers: the Global Knowledge Prior Capturer (GKPC) and the Local Region Subject Capturer (LRSC), which respectively capture global body knowledge and local joint information. Additionally, we propose three joint constraint mechanisms to express the potential association dependencies between global and local joints, which are further modeled using two association capturers: the Refined-Regression Association Capture Module (RR-ACM) and the Generalized-Guidance Association Capture Module (GG-ACM). Moreover, we introduce a novel feature transformation module, the Link Conversion Module (LCM), to transform and augment pose features. The algorithm adopts a complementary process to enhance the interaction and fusion of global and local feature information by gradually imposing constraints on the physical topological features of the human body, thereby improving its modeling capabilities. Extensive experiments demonstrate that our proposed ICFNet achieves state-of-the-art results on two challenging benchmark datasets: Human 3.6M and MPI-INF-3DHP. The code and model are available at: <span><span>https://github.com/PENG-LAU/ICFNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128947"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing object recognition: The role of object knowledge decomposition and component-labeled datasets 增强对象识别：对象知识分解和组件标记数据集的作用

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128969

Nuoye Xiong , Ning Wang , Hongsheng Li , Guangming Zhu , Liang Zhang , Syed Afaq Ali Shah , Mohammed Bennamoun

Deep learning models’ decision-making processes can be elusive, often raising concerns about their reliability. To address this, we have introduced the Object Knowledge Decomposition and Components Label Dataset (OKD-CL), designed to improve the interpretability and accuracy of object recognition models. This dataset includes 99 categories from PartImageNet, each detailed with clear physical structures that align with human visual concepts. In a hierarchical structure, every category is described by Abstract Component Knowledge (ACK) descriptions and each image instance comes with Explicit Visual Knowledge (EVK) masks, highlighting the visual components’ appearance. By evaluating multiple deep neural networks guided with ACK and EVK (dual-knowledge-guidance approach), we saw better accuracy and a higher Foreground Reasoning Ratio (FRR), confirming our knowledge-guided method’s effectiveness. When used on the Hard-ImageNet dataset, this approach reduced the model’s reliance on incorrect feature assumptions without sacrificing classification accuracy. This hierarchical comprehension encouraged by OKD-CL is crucial in minimizing incorrect feature associations and strengthening model robustness. The entire code and dataset are available on: https://github.com/XiGuaBo/OKD-CL.

深度学习模型的决策过程可能难以捉摸，这常常引发人们对其可靠性的担忧。为了解决这个问题，我们引入了对象知识分解和组件标签数据集（OKD-CL），旨在提高对象识别模型的可解释性和准确性。该数据集包括来自PartImageNet的99个类别，每个类别都有清晰的物理结构，与人类的视觉概念保持一致。在层次结构中，每个类别都通过抽象组件知识（ACK）描述来描述，每个图像实例都带有显式视觉知识（EVK）掩码，突出显示视觉组件的外观。通过对ACK和EVK（双知识引导方法）引导的多个深度神经网络进行评估，我们看到了更好的准确率和更高的前景推理比率（FRR），证实了我们的知识引导方法的有效性。当在Hard-ImageNet数据集上使用时，这种方法减少了模型对不正确特征假设的依赖，而不会牺牲分类精度。OKD-CL鼓励的这种分层理解对于最小化不正确的特征关联和增强模型鲁棒性至关重要。完整的代码和数据集可在：https://github.com/XiGuaBo/OKD-CL。

{"title":"Enhancing object recognition: The role of object knowledge decomposition and component-labeled datasets","authors":"Nuoye Xiong , Ning Wang , Hongsheng Li , Guangming Zhu , Liang Zhang , Syed Afaq Ali Shah , Mohammed Bennamoun","doi":"10.1016/j.neucom.2024.128969","DOIUrl":"10.1016/j.neucom.2024.128969","url":null,"abstract":"<div><div>Deep learning models’ decision-making processes can be elusive, often raising concerns about their reliability. To address this, we have introduced the Object Knowledge Decomposition and Components Label Dataset (OKD-CL), designed to improve the interpretability and accuracy of object recognition models. This dataset includes 99 categories from PartImageNet, each detailed with clear physical structures that align with human visual concepts. In a hierarchical structure, every category is described by Abstract Component Knowledge (ACK) descriptions and each image instance comes with Explicit Visual Knowledge (EVK) masks, highlighting the visual components’ appearance. By evaluating multiple deep neural networks guided with ACK and EVK (dual-knowledge-guidance approach), we saw better accuracy and a higher Foreground Reasoning Ratio (FRR), confirming our knowledge-guided method’s effectiveness. When used on the Hard-ImageNet dataset, this approach reduced the model’s reliance on incorrect feature assumptions without sacrificing classification accuracy. This hierarchical comprehension encouraged by OKD-CL is crucial in minimizing incorrect feature associations and strengthening model robustness. The entire code and dataset are available on: <span><span>https://github.com/XiGuaBo/OKD-CL</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"617 ","pages":"Article 128969"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142745307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TCM: An efficient lightweight MLP-based network with affine transformation for long-term time series forecasting 基于仿射变换的高效轻量级mlp网络，用于长期时间序列预测

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128960

Hongwei Jiang, Dongsheng Liu, Xinyi Ding, Yaning Chen, Hongtao Li

Time series forecasting (TSF) involves extracting underlying patterns from past information to predict future sequences over a specific period. Extending the prediction length of time series and improving the prediction accuracy have always been challenging tasks. Autoregressive prediction methods based on Markov chains tend to accumulate errors over time. Although Transformer-based methods with various self-attention mechanisms have shown some improvements, they require higher memory and computational resources. In this work, we present an effective MLP-based TSF framework named TCM, which models the sequence and channel dependencies separately using Token MLP and Channel MLP. Additionally, we employ the Affine Transformation to replace layer normalization or batch normalization, leading to substantial enhancements in both accuracy and inference speed. Compared to current state-of-the-art long-term time series forecasting models, TCM achieves 6.0% relative improvement on seven real-world datasets, including electricity, weather, and illness domains. The TCM model, characterized by its efficiency and lightweight architecture, also makes it suitable for scenarios with high real-time requirements.

时间序列预测（TSF）涉及从过去信息中提取潜在模式，以预测特定时期内的未来序列。延长时间序列的预测长度，提高预测精度一直是具有挑战性的课题。基于马尔可夫链的自回归预测方法往往会随着时间的推移而累积误差。尽管具有各种自关注机制的基于transformer的方法已经显示出一些改进，但它们需要更高的内存和计算资源。在这项工作中，我们提出了一个有效的基于MLP的TSF框架TCM，它使用Token MLP和channel MLP分别对序列和通道依赖关系进行建模。此外，我们采用仿射变换来取代层归一化或批归一化，从而大大提高了准确性和推理速度。与目前最先进的长期时间序列预测模型相比，TCM在包括电力、天气和疾病领域在内的七个现实世界数据集上实现了6.0%的相对改进。TCM模型具有高效、轻量化的特点，也适用于实时性要求高的场景。

{"title":"TCM: An efficient lightweight MLP-based network with affine transformation for long-term time series forecasting","authors":"Hongwei Jiang, Dongsheng Liu, Xinyi Ding, Yaning Chen, Hongtao Li","doi":"10.1016/j.neucom.2024.128960","DOIUrl":"10.1016/j.neucom.2024.128960","url":null,"abstract":"<div><div>Time series forecasting (TSF) involves extracting underlying patterns from past information to predict future sequences over a specific period. Extending the prediction length of time series and improving the prediction accuracy have always been challenging tasks. Autoregressive prediction methods based on Markov chains tend to accumulate errors over time. Although Transformer-based methods with various self-attention mechanisms have shown some improvements, they require higher memory and computational resources. In this work, we present an effective MLP-based TSF framework named TCM, which models the sequence and channel dependencies separately using Token MLP and Channel MLP. Additionally, we employ the Affine Transformation to replace layer normalization or batch normalization, leading to substantial enhancements in both accuracy and inference speed. Compared to current state-of-the-art long-term time series forecasting models, TCM achieves 6.0% relative improvement on seven real-world datasets, including electricity, weather, and illness domains. The TCM model, characterized by its efficiency and lightweight architecture, also makes it suitable for scenarios with high real-time requirements.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"617 ","pages":"Article 128960"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142745435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Novelty-aware concept drift detection for neural networks 神经网络的新奇感知概念漂移检测

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-22 DOI: 10.1016/j.neucom.2024.128933

Dan Shang, Guangquan Zhang, Jie Lu

Neural network models are widely adopted in real-world applications for processing streaming data. However, these applications often face challenges in terms of accuracy degradation, caused by changes in the data distribution of the stream data compared to the training data. Two underlying reasons contribute to these changes. The first, known as the concept drift problem, occurs when there is a change in the correlation between the input data and the prediction output, making the models trained on the training data no longer suitable for the new data. The second reason, known as the novelty problem, arises when real-world data contains unexpected data categories that were not present in the training data, resulting in incorrect predictions. The research community has divided into different groups and each developed various methods to detect either concept drift or novelty distribution changes. However, these methods only address one aspect of the problem and are unable to distinguish between them. This leads to an inappropriate allocation of model maintenance resources, including the high cost of model retraining and the acquisition of true label data. In this study, we aim to address this gap by proposing a novel concept drift detection method that is capable of distinguishing between known labeled concept drift and novelty. Our method is also more efficient than existing drift detection methods, making it suitable for applications on neural networks.

神经网络模型在处理流数据的实际应用中被广泛采用。然而，这些应用程序经常面临准确性下降的挑战，这是由流数据与训练数据的数据分布的变化引起的。两个潜在的原因导致了这些变化。第一种是概念漂移问题，当输入数据和预测输出之间的相关性发生变化，使得在训练数据上训练的模型不再适合新数据时，就会发生概念漂移问题。第二个原因被称为新颖性问题，当真实数据包含训练数据中不存在的意外数据类别时，就会出现不正确的预测。研究界已经分成了不同的小组，每个小组都开发了各种方法来检测概念漂移或新颖性分布的变化。然而，这些方法只解决问题的一个方面，无法区分它们。这将导致模型维护资源的不适当分配，包括模型再训练和获取真实标签数据的高成本。在本研究中，我们的目标是通过提出一种新的概念漂移检测方法来解决这一差距，该方法能够区分已知标记概念漂移和新颖性。我们的方法也比现有的漂移检测方法更有效，使其适合于神经网络的应用。

{"title":"Novelty-aware concept drift detection for neural networks","authors":"Dan Shang, Guangquan Zhang, Jie Lu","doi":"10.1016/j.neucom.2024.128933","DOIUrl":"10.1016/j.neucom.2024.128933","url":null,"abstract":"<div><div>Neural network models are widely adopted in real-world applications for processing streaming data. However, these applications often face challenges in terms of accuracy degradation, caused by changes in the data distribution of the stream data compared to the training data. Two underlying reasons contribute to these changes. The first, known as the concept drift problem, occurs when there is a change in the correlation between the input data and the prediction output, making the models trained on the training data no longer suitable for the new data. The second reason, known as the novelty problem, arises when real-world data contains unexpected data categories that were not present in the training data, resulting in incorrect predictions. The research community has divided into different groups and each developed various methods to detect either concept drift or novelty distribution changes. However, these methods only address one aspect of the problem and are unable to distinguish between them. This leads to an inappropriate allocation of model maintenance resources, including the high cost of model retraining and the acquisition of true label data. In this study, we aim to address this gap by proposing a novel concept drift detection method that is capable of distinguishing between known labeled concept drift and novelty. Our method is also more efficient than existing drift detection methods, making it suitable for applications on neural networks.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"617 ","pages":"Article 128933"},"PeriodicalIF":5.5,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142745444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Newton cooling theorem-based local overlapping regions cleaning and oversampling techniques for imbalanced datasets 基于牛顿冷却定理的不平衡数据集局部重叠区域清理和过采样技术

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-21 DOI: 10.1016/j.neucom.2024.128959

Liangliang Tao , Qingya Wang , Fen Yu , Hui Cao , Yage Liang , Huixia Luo , Jinghui Guo

Imbalanced datasets pose significant challenges to machine learning tasks because traditional classifiers tend to favor the majority class. While numerous methods have been proposed to balance data distribution, recent studies have identified that imbalanced classification is also hindered by other data characteristics. Among these factors, the joint effects of class overlap and within-class imbalance are particularly harmful to classification performance. To the end, we propose a novel algorithm called Newton Cooling Theorem-Based Local Overlapping Regions Cleaning and Oversampling (NCLO-SMOTE). This method employs an adaptive semi-supervised clustering algorithm, which divides the minority class into several clusters without requiring a pre-set number of clusters. It quantifies both the overall and local overlapping degrees of the dataset based on the number of clusters and their local information. Additionally, it uses Newton’s Cooling Theorem to clean these overlapping regions and a cluster-weighted oversampling strategy to address within-class imbalance. Comparative experiments were conducted between NCLO-SMOTE and ten state-of-the-art sampling methods on 48 real-world imbalanced datasets. The experimental results demonstrate that our proposed method not only achieves superior performance but also exhibits strong robustness and versatility in handling the joint effects of class overlap and imbalance.

不平衡的数据集给机器学习任务带来了重大挑战，因为传统的分类器倾向于支持大多数类别。虽然已经提出了许多方法来平衡数据分布，但最近的研究发现，不平衡分类也受到其他数据特征的阻碍。在这些因素中，类重叠和类内不平衡的共同作用对分类性能的影响尤为严重。最后，我们提出了一种基于牛顿冷却定理的局部重叠区域清洗和过采样（ncloo - smote）算法。该方法采用自适应半监督聚类算法，在不需要预先设定簇数的情况下，将少数类划分为若干个簇。它根据聚类的数量及其局部信息量化数据集的整体和局部重叠程度。此外，它使用牛顿冷却定理来清理这些重叠区域，并使用聚类加权过采样策略来解决类内不平衡问题。在48个真实不平衡数据集上，将ncloo - smote与10种最先进的采样方法进行了对比实验。实验结果表明，该方法不仅具有较好的性能，而且在处理类重叠和类不平衡的联合效应方面具有较强的鲁棒性和通用性。

{"title":"Newton cooling theorem-based local overlapping regions cleaning and oversampling techniques for imbalanced datasets","authors":"Liangliang Tao , Qingya Wang , Fen Yu , Hui Cao , Yage Liang , Huixia Luo , Jinghui Guo","doi":"10.1016/j.neucom.2024.128959","DOIUrl":"10.1016/j.neucom.2024.128959","url":null,"abstract":"<div><div>Imbalanced datasets pose significant challenges to machine learning tasks because traditional classifiers tend to favor the majority class. While numerous methods have been proposed to balance data distribution, recent studies have identified that imbalanced classification is also hindered by other data characteristics. Among these factors, the joint effects of class overlap and within-class imbalance are particularly harmful to classification performance. To the end, we propose a novel algorithm called <em>Newton Cooling Theorem-Based Local Overlapping Regions Cleaning and Oversampling</em> (NCLO-SMOTE). This method employs an adaptive semi-supervised clustering algorithm, which divides the minority class into several clusters without requiring a pre-set number of clusters. It quantifies both the overall and local overlapping degrees of the dataset based on the number of clusters and their local information. Additionally, it uses Newton’s Cooling Theorem to clean these overlapping regions and a cluster-weighted oversampling strategy to address within-class imbalance. Comparative experiments were conducted between NCLO-SMOTE and ten state-of-the-art sampling methods on 48 real-world imbalanced datasets. The experimental results demonstrate that our proposed method not only achieves superior performance but also exhibits strong robustness and versatility in handling the joint effects of class overlap and imbalance.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128959"},"PeriodicalIF":5.5,"publicationDate":"2024-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fewer interpretable bands via self-supervision for hyperspectral anomaly detection 在高光谱异常检测中，通过自我监督可解释的频带较少

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-21 DOI: 10.1016/j.neucom.2024.128964

Ruike Wang, Jing Hu

Hyperspectral band selection (BS) algorithms aim to identify the most important bands relevant to model decisions. While existing methods can effectively assign importance to different bands, they face challenges in unsupervised hyperspectral anomaly detection (AD) due to the lack of predefined labels to guide the selection process. This paper addresses these challenges by proposing a self-supervised approach for identifying fewer interpretable bands that best separate background and anomalous data. By formulating band selection as a subset selection problem using pseudo-anomalies, bands are chosen based on high confidence, rich feature diversity, and inter-collaboration. Specifically, the pseudo-anomalies and the corresponding map are utilized to identify bands with high confidence through a self-supervised strategy. To further enhance feature diversity and collaboration among bands, we impose a feature diversity constraint on the selection of subsets and assess the collaboration ability of various subsets. A novel evaluation function is designed to discover more useful bands for AD. Experiments demonstrate the effectiveness of each module and show that the proposed method selects bands that are beneficial for AD across four different datasets. The code is released at https://github.com/rk-rkk/Fewer-Interpretable-Bands-for-Hyperspectral-Anomaly-Detection.

高光谱波段选择（BS）算法旨在识别与模型决策相关的最重要波段。虽然现有方法可以有效地对不同波段进行重要性分配，但由于缺乏预定义标签来指导选择过程，因此在无监督高光谱异常检测（AD）中面临挑战。本文通过提出一种自我监督的方法来解决这些挑战，该方法用于识别较少的可解释波段，从而最好地分离背景和异常数据。通过将波段选择描述为基于伪异常的子集选择问题，基于高置信度、丰富的特征多样性和相互协作来选择波段。具体来说，利用伪异常和相应的地图，通过自监督策略来识别高置信度的波段。为了进一步增强波段之间的特征多样性和协同性，我们对子集的选择施加了特征多样性约束，并对各子集的协同能力进行了评估。设计了一种新的评价函数来发现更多有用的AD波段。实验证明了每个模块的有效性，并表明该方法在四个不同的数据集上选择了有利于AD的频带。该代码发布在https://github.com/rk-rkk/Fewer-Interpretable-Bands-for-Hyperspectral-Anomaly-Detection。

{"title":"Fewer interpretable bands via self-supervision for hyperspectral anomaly detection","authors":"Ruike Wang, Jing Hu","doi":"10.1016/j.neucom.2024.128964","DOIUrl":"10.1016/j.neucom.2024.128964","url":null,"abstract":"<div><div>Hyperspectral band selection (BS) algorithms aim to identify the most important bands relevant to model decisions. While existing methods can effectively assign importance to different bands, they face challenges in unsupervised hyperspectral anomaly detection (AD) due to the lack of predefined labels to guide the selection process. This paper addresses these challenges by proposing a self-supervised approach for identifying fewer interpretable bands that best separate background and anomalous data. By formulating band selection as a subset selection problem using pseudo-anomalies, bands are chosen based on high confidence, rich feature diversity, and inter-collaboration. Specifically, the pseudo-anomalies and the corresponding map are utilized to identify bands with high confidence through a self-supervised strategy. To further enhance feature diversity and collaboration among bands, we impose a feature diversity constraint on the selection of subsets and assess the collaboration ability of various subsets. A novel evaluation function is designed to discover more useful bands for AD. Experiments demonstrate the effectiveness of each module and show that the proposed method selects bands that are beneficial for AD across four different datasets. The code is released at <span><span>https://github.com/rk-rkk/Fewer-Interpretable-Bands-for-Hyperspectral-Anomaly-Detection</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"616 ","pages":"Article 128964"},"PeriodicalIF":5.5,"publicationDate":"2024-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0