Neurocomputing最新文献_第5页

Sparse discriminant manifold projections for automatic depression recognition 用于自动抑郁识别的稀疏判别流形投影

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-04 DOI: 10.1016/j.neucom.2024.128765

Lu Zhang , Jitao Zhong , Qinglin Zhao , Shi Qiao , Yushan Wu , Bin Hu , Sujie Ma , Hong Peng

In recent years, depression has become an increasingly serious problem globally. Previous research have shown that EEG-based depression recognition is a promising technique to serve as auxiliary diagnosis methods that provide assistance to clinicians. Typically, in clinical studies, due to the multichannel nature of EEG, the extracted features usually are high-dimensional and contain many redundant information. Therefore, it is necessary to perform dimensionality reduction before classification to improve the performance of machine learning algorithms. However,existing dimensionality reduction techniques do not design the objective function based on the characteristics of EEG signal and the goal of depression recognition, so they are less suitable for dimensionality reduction of EEG features. To solve this problem, in this paper we propose a novel dimensionality reduction technique called sparse discriminant manifold projections(SDMP) for depression recognition. Specifically, the use of the

ℓ_{2}

-norm instead of the squared

ℓ_{2}

-norm as a similarity measure in the objective function reduces sensitivity to noise and outliers. Moreover, the local geometric structure and global discriminative properties of data are integrated, which makes the extracted features more discriminative. Finally, the

ℓ_{2, 1}

-norm regularization is introduced to achieve feature selection. Furthermore, The formulation is extended to the

ℓ_{2, p}

-norm regularization case, which is more likely to offer better sparsity when

0 < p < 1

. Extensive experiments on EEG data show that the SDMP achieves the competitive performance compared with other state-of-the-art dimensionality reduction methods. It also shows the practical application value of our method in detecting depression.

近年来，抑郁症已成为全球日益严重的问题。以往的研究表明，基于脑电图的抑郁症识别是一种很有前景的技术，可作为辅助诊断方法为临床医生提供帮助。通常，在临床研究中，由于脑电图的多通道特性，提取的特征通常是高维的，包含许多冗余信息。因此，有必要在分类前进行降维处理，以提高机器学习算法的性能。然而，现有的降维技术并没有根据脑电信号的特点和抑郁症识别的目标来设计目标函数，因此不太适合对脑电信号特征进行降维。为解决这一问题，本文提出了一种新型降维技术，即用于抑郁症识别的稀疏判别流形投影（SDMP）。具体来说，在目标函数中使用ℓ2-norm 而不是平方ℓ2-norm 作为相似性度量，降低了对噪声和异常值的敏感性。此外，数据的局部几何结构和全局判别特性被整合在一起，这使得提取的特征更具判别性。最后，引入 ℓ2,1 正则化来实现特征选择。在脑电图数据上的大量实验表明，与其他最先进的降维方法相比，SDMP 的性能更具竞争力。这也显示了我们的方法在检测抑郁症方面的实际应用价值。

{"title":"Sparse discriminant manifold projections for automatic depression recognition","authors":"Lu Zhang , Jitao Zhong , Qinglin Zhao , Shi Qiao , Yushan Wu , Bin Hu , Sujie Ma , Hong Peng","doi":"10.1016/j.neucom.2024.128765","DOIUrl":"10.1016/j.neucom.2024.128765","url":null,"abstract":"<div><div>In recent years, depression has become an increasingly serious problem globally. Previous research have shown that EEG-based depression recognition is a promising technique to serve as auxiliary diagnosis methods that provide assistance to clinicians. Typically, in clinical studies, due to the multichannel nature of EEG, the extracted features usually are high-dimensional and contain many redundant information. Therefore, it is necessary to perform dimensionality reduction before classification to improve the performance of machine learning algorithms. However,existing dimensionality reduction techniques do not design the objective function based on the characteristics of EEG signal and the goal of depression recognition, so they are less suitable for dimensionality reduction of EEG features. To solve this problem, in this paper we propose a novel dimensionality reduction technique called sparse discriminant manifold projections(SDMP) for depression recognition. Specifically, the use of the <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>2</mn></mrow></msub></math></span>-norm instead of the squared <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>2</mn></mrow></msub></math></span>-norm as a similarity measure in the objective function reduces sensitivity to noise and outliers. Moreover, the local geometric structure and global discriminative properties of data are integrated, which makes the extracted features more discriminative. Finally, the <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></math></span>-norm regularization is introduced to achieve feature selection. Furthermore, The formulation is extended to the <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>2</mn><mo>,</mo><mi>p</mi></mrow></msub></math></span>-norm regularization case, which is more likely to offer better sparsity when <span><math><mrow><mn>0</mn><mo><</mo><mi>p</mi><mo><</mo><mn>1</mn></mrow></math></span>. Extensive experiments on EEG data show that the SDMP achieves the competitive performance compared with other state-of-the-art dimensionality reduction methods. It also shows the practical application value of our method in detecting depression.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128765"},"PeriodicalIF":5.5,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An efficient re-parameterization feature pyramid network on YOLOv8 to the detection of steel surface defect 基于 YOLOv8 的高效重参数化特征金字塔网络用于检测钢铁表面缺陷

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-04 DOI: 10.1016/j.neucom.2024.128775

Weining Xie, Weifeng Ma, Xiaoyong Sun

In the field of steel production, the detection of steel surface defects is one of the most important guarantees for the quality of steel production. In the process of defect detection, there are problems regarding the noise of the acquisition background, the scale of defects, and the detection speed. At present, in the face of complex steel surface defects, realizing efficient real-time steel surface defect detection has become a difficult problem. In this paper, we propose a lightweight and efficient real-time defect detection method, LDE-YOLO, based on YOLOv8. First, we propose a lightweight multi-scale feature extraction module, LighterMSMC, which not only achieves a lightweight backbone network, but also effectively guarantees the long range dependence of the features, so as to realize multi-scale feature extraction more efficiently. Secondly, we propose lightweight re-parameterized feature pyramid, DE-FPN, in which the sparse patterns of the overall features and the detailed features of the local features are efficiently captured by the DE-Block, and then efficiently fused by the PAN feature fusion structure. Finally, we propose Efficient Head, which lightens the model by group convolution while its improves the diagonal correlation of the feature maps on some specific datasets, thus enhancing the detection performance. Our proposed LDE-YOLO obtains 80.8 mAP and 75.5 FPS on NEU-DET , 80.5 mAP and 75.5 FPS on GC10-DET. It obtains 2.5 mAP and 4.7 mAP enhancement compared to the baseline model, and the detection speed is also improved by 10.4 FPS, while in terms of the number of floating point operations and parameters of the model reduced by 60.2% and 49.1%, which is sufficient to illustrate its lightweight effectiveness and realize an efficient real-time steel surface defect detection model.

在钢铁生产领域，钢铁表面缺陷的检测是钢铁生产质量的重要保证之一。在缺陷检测过程中，存在采集背景噪声、缺陷尺度、检测速度等问题。目前，面对复杂的钢材表面缺陷，实现高效的实时钢材表面缺陷检测已成为一个难题。本文在 YOLOv8 的基础上，提出了一种轻量级高效实时缺陷检测方法 LDE-YOLO。首先，我们提出了轻量级多尺度特征提取模块 LighterMSMC，不仅实现了骨干网络的轻量级，还有效保证了特征的远距离依赖性，从而更高效地实现多尺度特征提取。其次，我们提出了轻量级重参数化特征金字塔 DE-FPN，通过 DE-Block 有效捕捉整体特征的稀疏模式和局部特征的细节特征，再通过 PAN 特征融合结构进行高效融合。最后，我们提出了 Efficient Head，它通过群卷积来简化模型，同时在一些特定数据集上改进了特征图的对角相关性，从而提高了检测性能。我们提出的 LDE-YOLO 在 NEU-DET 上获得了 80.8 mAP 和 75.5 FPS，在 GC10-DET 上获得了 80.5 mAP 和 75.5 FPS。与基线模型相比，分别提高了 2.5 mAP 和 4.7 mAP，检测速度也提高了 10.4 FPS，同时模型的浮点运算次数和参数分别减少了 60.2% 和 49.1%，足以说明其轻量化的有效性，实现了高效的实时钢表面缺陷检测模型。

{"title":"An efficient re-parameterization feature pyramid network on YOLOv8 to the detection of steel surface defect","authors":"Weining Xie, Weifeng Ma, Xiaoyong Sun","doi":"10.1016/j.neucom.2024.128775","DOIUrl":"10.1016/j.neucom.2024.128775","url":null,"abstract":"<div><div>In the field of steel production, the detection of steel surface defects is one of the most important guarantees for the quality of steel production. In the process of defect detection, there are problems regarding the noise of the acquisition background, the scale of defects, and the detection speed. At present, in the face of complex steel surface defects, realizing efficient real-time steel surface defect detection has become a difficult problem. In this paper, we propose a lightweight and efficient real-time defect detection method, LDE-YOLO, based on YOLOv8. First, we propose a lightweight multi-scale feature extraction module, LighterMSMC, which not only achieves a lightweight backbone network, but also effectively guarantees the long range dependence of the features, so as to realize multi-scale feature extraction more efficiently. Secondly, we propose lightweight re-parameterized feature pyramid, DE-FPN, in which the sparse patterns of the overall features and the detailed features of the local features are efficiently captured by the DE-Block, and then efficiently fused by the PAN feature fusion structure. Finally, we propose Efficient Head, which lightens the model by group convolution while its improves the diagonal correlation of the feature maps on some specific datasets, thus enhancing the detection performance. Our proposed LDE-YOLO obtains 80.8 mAP and 75.5 FPS on NEU-DET , 80.5 mAP and 75.5 FPS on GC10-DET. It obtains 2.5 mAP and 4.7 mAP enhancement compared to the baseline model, and the detection speed is also improved by 10.4 FPS, while in terms of the number of floating point operations and parameters of the model reduced by 60.2% and 49.1%, which is sufficient to illustrate its lightweight effectiveness and realize an efficient real-time steel surface defect detection model.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128775"},"PeriodicalIF":5.5,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142592914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-contrast image clustering via multi-resolution augmentation and momentum-output queues 通过多分辨率增强和动量输出队列进行多对比度图像聚类

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-01 DOI: 10.1016/j.neucom.2024.128738

Sheng Jin, Shuisheng Zhou, Dezheng Kong, Banghe Han

Contrastive clustering has emerged as an efficacious technique in the domain of deep clustering, leveraging the interplay between paired samples and the learning capabilities of deep network architectures. However, the augmentation strategies employed in the existing methods do not fully utilize the information of images, coupled with the limitation of the number of negative samples makes the clustering performance suffer. In this study, we propose a novel clustering approach that incorporates momentum-output queues and multi-resolution augmentation strategies to effectively address these limitations. Initially, we deploy a multi-resolution augmentation strategy, transforming conventional augmentations into distinct global and local perspectives across various resolutions. This approach comprehensively harnesses inter-image information to construct a multi-contrast model with multi-view inputs. Subsequently, we introduce momentum-output queues, which are designed to store a large number of negative samples without increasing the computational cost, thereby enhancing the clustering effect. Within our joint optimization framework, sample features are derived from both the original and momentum encoders for instance-level contrastive learning. Simultaneously, features produced exclusively by the original encoder within the same batch are employed for cluster-level contrastive learning. Our experimental results on five challenging datasets substantiate the superior performance of our method over existing state-of-the-art techniques.

对比聚类利用配对样本之间的相互作用和深度网络架构的学习能力，已成为深度聚类领域的一种有效技术。然而，现有方法采用的增强策略并不能充分利用图像信息，再加上负样本数量的限制，使得聚类性能大打折扣。在本研究中，我们提出了一种结合动量输出队列和多分辨率增强策略的新型聚类方法，以有效解决这些局限性。首先，我们部署了一种多分辨率增强策略，将传统的增强转化为不同分辨率的全局和局部视角。这种方法可全面利用图像间信息，构建具有多视角输入的多对比度模型。随后，我们引入了动量输出队列，旨在存储大量负样本而不增加计算成本，从而增强聚类效果。在我们的联合优化框架内，样本特征来自于原始编码器和动量编码器，用于实例级对比学习。与此同时，同一批次中完全由原始编码器生成的特征被用于集群级对比学习。我们在五个具有挑战性的数据集上的实验结果证明，我们的方法比现有的最先进技术性能更优越。

{"title":"Multi-contrast image clustering via multi-resolution augmentation and momentum-output queues","authors":"Sheng Jin, Shuisheng Zhou, Dezheng Kong, Banghe Han","doi":"10.1016/j.neucom.2024.128738","DOIUrl":"10.1016/j.neucom.2024.128738","url":null,"abstract":"<div><div>Contrastive clustering has emerged as an efficacious technique in the domain of deep clustering, leveraging the interplay between paired samples and the learning capabilities of deep network architectures. However, the augmentation strategies employed in the existing methods do not fully utilize the information of images, coupled with the limitation of the number of negative samples makes the clustering performance suffer. In this study, we propose a novel clustering approach that incorporates momentum-output queues and multi-resolution augmentation strategies to effectively address these limitations. Initially, we deploy a multi-resolution augmentation strategy, transforming conventional augmentations into distinct global and local perspectives across various resolutions. This approach comprehensively harnesses inter-image information to construct a multi-contrast model with multi-view inputs. Subsequently, we introduce momentum-output queues, which are designed to store a large number of negative samples without increasing the computational cost, thereby enhancing the clustering effect. Within our joint optimization framework, sample features are derived from both the original and momentum encoders for instance-level contrastive learning. Simultaneously, features produced exclusively by the original encoder within the same batch are employed for cluster-level contrastive learning. Our experimental results on five challenging datasets substantiate the superior performance of our method over existing state-of-the-art techniques.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128738"},"PeriodicalIF":5.5,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142592863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PIDNODEs: Neural ordinary differential equations inspired by a proportional–integral–derivative controller PIDNODEs：受比例-积分-派生控制器启发的神经常微分方程

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-30 DOI: 10.1016/j.neucom.2024.128769

Pengkai Wang , Song Chen , Jiaxu Liu , Shengze Cai , Chao Xu

Neural Ordinary Differential Equations (NODEs) are a novel family of infinite-depth neural-net models through solving ODEs and their adjoint equations. In this paper, we present a strategy to enhance the training and inference of NODEs by integrating a Proportional–Integral–Derivative (PID) controller into the framework of Heavy Ball NODE, resulting in the proposed PIDNODEs and its generalized version, GPIDNODEs. By leveraging the advantages of control, PIDNODEs and GPIDNODEs can address the stiff ODE challenges by adjusting the parameters (i.e.,

K_{p}

,

K_{i}

and

K_{d}

) in the PID module. The experiments confirm the superiority of PIDNODEs/GPIDNODEs over other NODE baselines on different computer vision and pattern recognition tasks, including image classification, point cloud separation and learning long-term dependencies from irregular time-series data for a physical dynamic system. These experiments demonstrate that the proposed models have higher accuracy and fewer function evaluations while alleviating the dilemma of exploding and vanishing gradients, particularly when learning long-term dependencies from a large amount of data.

神经常微分方程（NODEs）是通过求解 ODEs 及其邻接方程来建立无限深度神经网络模型的新型系列。在本文中，我们提出了一种增强 NODEs 训练和推理的策略，即在重球 NODE 框架中集成比例-正积分-反演 (PID) 控制器，从而形成了 PIDNODEs 及其广义版本 GPIDNODEs。PIDNODEs 和 GPIDNODEs 利用控制的优势，通过调整 PID 模块中的参数（即 Kp、Ki 和 Kd），可以解决僵化的 ODE 挑战。实验证实，在不同的计算机视觉和模式识别任务中，PIDNODEs/GPIDNODEs 优于其他 NODE 基线，这些任务包括图像分类、点云分离以及从物理动态系统的不规则时间序列数据中学习长期依赖关系。这些实验证明，所提出的模型具有更高的准确性和更少的函数评估次数，同时缓解了梯度爆炸和消失的困境，特别是从大量数据中学习长期依赖关系时更是如此。

{"title":"PIDNODEs: Neural ordinary differential equations inspired by a proportional–integral–derivative controller","authors":"Pengkai Wang , Song Chen , Jiaxu Liu , Shengze Cai , Chao Xu","doi":"10.1016/j.neucom.2024.128769","DOIUrl":"10.1016/j.neucom.2024.128769","url":null,"abstract":"<div><div>Neural Ordinary Differential Equations (NODEs) are a novel family of infinite-depth neural-net models through solving ODEs and their adjoint equations. In this paper, we present a strategy to enhance the training and inference of NODEs by integrating a Proportional–Integral–Derivative (PID) controller into the framework of Heavy Ball NODE, resulting in the proposed PIDNODEs and its generalized version, GPIDNODEs. By leveraging the advantages of control, PIDNODEs and GPIDNODEs can address the stiff ODE challenges by adjusting the parameters (i.e., <span><math><msub><mrow><mi>K</mi></mrow><mrow><mi>p</mi></mrow></msub></math></span>, <span><math><msub><mrow><mi>K</mi></mrow><mrow><mi>i</mi></mrow></msub></math></span> and <span><math><msub><mrow><mi>K</mi></mrow><mrow><mi>d</mi></mrow></msub></math></span>) in the PID module. The experiments confirm the superiority of PIDNODEs/GPIDNODEs over other NODE baselines on different computer vision and pattern recognition tasks, including image classification, point cloud separation and learning long-term dependencies from irregular time-series data for a physical dynamic system. These experiments demonstrate that the proposed models have higher accuracy and fewer function evaluations while alleviating the dilemma of exploding and vanishing gradients, particularly when learning long-term dependencies from a large amount of data.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128769"},"PeriodicalIF":5.5,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Breaking the gap between label correlation and instance similarity via new multi-label contrastive learning 通过新型多标签对比学习打破标签相关性与实例相似性之间的差距

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-29 DOI: 10.1016/j.neucom.2024.128719

Xin Wang , Wang Zhang , Yuhong Wu , Xingpeng Zhang , Chao Wang , Huayi Zhan

Multi-label text classification (MLTC) is a fundamental yet challenging task in natural language processing. Existing MLTC models mostly learn text representations and label correlations, separately; while the instance-level correlation, which is crucial for the classification is ignored. To rectify this, we propose a new multi-label contrastive learning model, that captures instance-level correlations, for the MLTC task. Specifically, we first learn label representations by using Graph Convolutional Network (GCN) on label co-occurrence graphs. We next learn text representations by taking label correlations into consideration. Through an attention mechanism, instance-level correlation can be established. To better utilize label correlations, we propose a new contrastive learning model, whose learning is guided by a new learning objective, to further refine label representations. We finally implement a

k

-NN mechanism, that identifies

k

nearest neighbors of a given text for final prediction. Intensive experimental studies over benchmark multi-label datasets demonstrate the effectiveness of our approach.

多标签文本分类（MLTC）是自然语言处理中一项基本而又具有挑战性的任务。现有的多标签文本分类模型大多分别学习文本表征和标签相关性，而忽略了对分类至关重要的实例级相关性。为了纠正这一问题，我们针对 MLTC 任务提出了一种新的多标签对比学习模型，该模型能捕捉实例级相关性。具体来说，我们首先在标签共现图上使用图卷积网络（GCN）学习标签表示。接下来，我们通过考虑标签相关性来学习文本表征。通过注意机制，可以建立实例级相关性。为了更好地利用标签相关性，我们提出了一种新的对比学习模型，其学习由新的学习目标引导，以进一步完善标签表征。最后，我们实施了一种 k-NN 机制，该机制可识别给定文本的 k 个近邻以进行最终预测。对基准多标签数据集的深入实验研究证明了我们方法的有效性。

{"title":"Breaking the gap between label correlation and instance similarity via new multi-label contrastive learning","authors":"Xin Wang , Wang Zhang , Yuhong Wu , Xingpeng Zhang , Chao Wang , Huayi Zhan","doi":"10.1016/j.neucom.2024.128719","DOIUrl":"10.1016/j.neucom.2024.128719","url":null,"abstract":"<div><div>Multi-label text classification (MLTC) is a fundamental yet challenging task in natural language processing. Existing MLTC models mostly learn text representations and label correlations, separately; while the instance-level correlation, which is crucial for the classification is ignored. To rectify this, we propose a new multi-label contrastive learning model, that captures instance-level correlations, for the MLTC task. Specifically, we first learn label representations by using Graph Convolutional Network (GCN) on label co-occurrence graphs. We next learn text representations by taking label correlations into consideration. Through an attention mechanism, instance-level correlation can be established. To better utilize label correlations, we propose a new contrastive learning model, whose learning is guided by a new learning objective, to further refine label representations. We finally implement a <span><math><mi>k</mi></math></span>-NN mechanism, that identifies <span><math><mi>k</mi></math></span> nearest neighbors of a given text for final prediction. Intensive experimental studies over benchmark multi-label datasets demonstrate the effectiveness of our approach.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128719"},"PeriodicalIF":5.5,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Double machine learning for partially linear mediation models with high-dimensional confounders 具有高维混杂因素的部分线性中介模型的双重机器学习

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-29 DOI: 10.1016/j.neucom.2024.128766

Jichen Yang, Yujing Shao, Jin Liu, Lei Wang

To estimate and statistically infer the direct and indirect effects of exposure and mediator variables while accounting for high-dimensional confounding variables, we propose a partially linear mediation model to incorporate a flexible mechanism of confounders. To obtain asymptotically efficient estimators for the effects of interest under the influence of the nuisance functions with high-dimensional confounders, we construct two Neyman-orthogonal score functions to remove regularization bias. Flexible machine learning methods and data splitting with cross-fitting are employed to address the overfitting issue and estimate unknown nuisance functions efficiently. We rigorously investigate the asymptotic expressions of the proposed estimators for the direct, indirect and total effects and then derive their asymptotic normality properties. In addition, two Wald statistics are constructed to test the direct and indirect effects, respectively, and their limiting distributions are obtained. The satisfactory performance of our proposed estimators is demonstrated by simulation results and a genome-wide analysis of blood DNA methylation dataset.

为了估计和统计推断暴露变量和中介变量的直接和间接效应，同时考虑高维混杂变量，我们提出了一个部分线性中介模型，以纳入灵活的混杂因素机制。为了在高维混杂变量滋扰函数的影响下获得渐进有效的相关效应估计值，我们构建了两个奈曼正交得分函数来消除正则化偏差。我们采用灵活的机器学习方法和交叉拟合的数据拆分来解决过拟合问题，并高效地估计未知的滋扰函数。我们严格研究了所提出的直接效应、间接效应和总效应估计器的渐近表达式，然后推导出它们的渐近正态性质。此外，我们还构建了两个 Wald 统计量，分别用于检验直接效应和间接效应，并得到了它们的极限分布。模拟结果和对血液 DNA 甲基化数据集的全基因组分析表明，我们提出的估计值具有令人满意的性能。

{"title":"Double machine learning for partially linear mediation models with high-dimensional confounders","authors":"Jichen Yang, Yujing Shao, Jin Liu, Lei Wang","doi":"10.1016/j.neucom.2024.128766","DOIUrl":"10.1016/j.neucom.2024.128766","url":null,"abstract":"<div><div>To estimate and statistically infer the direct and indirect effects of exposure and mediator variables while accounting for high-dimensional confounding variables, we propose a partially linear mediation model to incorporate a flexible mechanism of confounders. To obtain asymptotically efficient estimators for the effects of interest under the influence of the nuisance functions with high-dimensional confounders, we construct two Neyman-orthogonal score functions to remove regularization bias. Flexible machine learning methods and data splitting with cross-fitting are employed to address the overfitting issue and estimate unknown nuisance functions efficiently. We rigorously investigate the asymptotic expressions of the proposed estimators for the direct, indirect and total effects and then derive their asymptotic normality properties. In addition, two Wald statistics are constructed to test the direct and indirect effects, respectively, and their limiting distributions are obtained. The satisfactory performance of our proposed estimators is demonstrated by simulation results and a genome-wide analysis of blood DNA methylation dataset.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128766"},"PeriodicalIF":5.5,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142593059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Bipartite containment tracking for nonlinear MASs under FDI attack based on model-free adaptive iterative learning control 基于无模型自适应迭代学习控制的 FDI 攻击下非线性 MAS 的两端遏制跟踪

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-29 DOI: 10.1016/j.neucom.2024.128783

Xinning He, Zhongsheng Hou

The bipartite containment control problem for a type of heterogeneous multi-agent systems (MASs) under false data injection (FDI) attack is handled in this work by using the distributed model-free adaptive iterative learning control scheme with attack compensation. The unknown non-affine nonlinear dynamics of each agent is first transformed into an equivalent attack-related data model along the iteration axis using a compact form dynamic linearization method. Then, a distributed model-free adaptive iterative learning bipartite containment control (DMFAILBCC) scheme is constructed by employing I/O data from MASs, and the convergence is proved by rigorous mathematical analysis In addition, the updated control method and the convergence analysis will be extended to iteration switching topologies. Finally, the performance of the two proposed schemes is validated through numerical simulations and comparisons with different control schemes.

本研究利用带攻击补偿的分布式无模型自适应迭代学习控制方案，解决了在虚假数据注入（FDI）攻击下异构多代理系统（MAS）的双向遏制控制问题。首先，利用紧凑形式动态线性化方法将每个代理的未知非参数非线性动态沿迭代轴转换为与攻击相关的等效数据模型。然后，利用 MAS 的 I/O 数据，构建了分布式无模型自适应迭代学习双分区包含控制（DMFAILBCC）方案，并通过严格的数学分析证明了其收敛性。最后，通过数值模拟以及与不同控制方案的比较，验证了两种拟议方案的性能。

引用次数: 0

Augmented ELBO regularization for enhanced clustering in variational autoencoders 增强 ELBO 正则化，提高变异自动编码器的聚类能力

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-29 DOI: 10.1016/j.neucom.2024.128795

Kwangtek Na , Ju-Hong Lee , Eunchan Kim

With significant advances in deep neural networks, various new algorithms have emerged that effectively model latent structures within data, surpassing traditional clustering methods. Each data point is expected to belong to a single cluster in a typical clustering algorithm. However, clustering based on variational autoencoders (VAEs) represents the expectation of the overall clusters, denoted as

c = 1, \dots, K

in the KL divergence term. Consequently, the latent embedding

z

can be learned to exist across multiple clusters with relatively balanced probabilities, rather than being strongly associated with a specific cluster. This study introduces an additional regularizer to encourage the latent embedding

z

to have a strong affiliation with specific clusters. We introduce optimization methods to maximize the ELBO that includes the newly added regularization term and explore methods to eliminate computationally challenging terms. The positive impact of this regularization on clustering accuracy was verified by examining the variance of the final cluster probabilities. Furthermore, an enhancement in the clustering performance was observed when regularization was introduced.

随着深度神经网络的长足发展，各种新算法应运而生，它们能有效地模拟数据中的潜在结构，超越了传统的聚类方法。在典型的聚类算法中，每个数据点都属于一个聚类。然而，基于变异自编码器（VAE）的聚类代表了整体聚类的期望值，在 KL 发散项中表示为 c=1,...,K。因此，可以学习到潜在嵌入 z 以相对均衡的概率存在于多个聚类中，而不是与特定聚类紧密相关。本研究引入了一个额外的正则因子，以鼓励潜在内嵌 z 与特定聚类有较强的关联。我们引入了优化方法来最大化包含新添加的正则项的 ELBO，并探索了消除计算上具有挑战性的项的方法。通过检查最终聚类概率的方差，验证了正则化对聚类准确性的积极影响。此外，引入正则化后，聚类性能也得到了提高。

{"title":"Augmented ELBO regularization for enhanced clustering in variational autoencoders","authors":"Kwangtek Na , Ju-Hong Lee , Eunchan Kim","doi":"10.1016/j.neucom.2024.128795","DOIUrl":"10.1016/j.neucom.2024.128795","url":null,"abstract":"<div><div>With significant advances in deep neural networks, various new algorithms have emerged that effectively model latent structures within data, surpassing traditional clustering methods. Each data point is expected to belong to a single cluster in a typical clustering algorithm. However, clustering based on variational autoencoders (VAEs) represents the expectation of the overall clusters, denoted as <span><math><mrow><mi>c</mi><mo>=</mo><mn>1</mn><mo>,</mo><mo>…</mo><mo>,</mo><mi>K</mi></mrow></math></span> in the KL divergence term. Consequently, the latent embedding <span><math><mi>z</mi></math></span> can be learned to exist across multiple clusters with relatively balanced probabilities, rather than being strongly associated with a specific cluster. This study introduces an additional regularizer to encourage the latent embedding <span><math><mi>z</mi></math></span> to have a strong affiliation with specific clusters. We introduce optimization methods to maximize the ELBO that includes the newly added regularization term and explore methods to eliminate computationally challenging terms. The positive impact of this regularization on clustering accuracy was verified by examining the variance of the final cluster probabilities. Furthermore, an enhancement in the clustering performance was observed when regularization was introduced.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128795"},"PeriodicalIF":5.5,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning from different perspectives for regret reduction in reinforcement learning: A free energy approach 从不同角度学习，减少强化学习中的遗憾：自由能方法

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-29 DOI: 10.1016/j.neucom.2024.128797

Milad Ghorbani, Reshad Hosseini, Seyed Pooya Shariatpanahi, Majid Nili Ahmadabadi

Reinforcement learning (RL) is the core method for interactive learning in living and artificial creatures. Nevertheless, in contrast to humans and animals, artificial RL agents are very slow in learning and suffer from the curse of dimensionality. This is partially due to using RL in isolation; i.e. lack of social learning and social diversity. We introduce a free energy-based social RL for learning novel tasks. Society is formed by the learning agent and some diverse virtual ones. That diversity is in their perception while all agents use the same interaction samples for learning and share the same action set. Individual difference in perception is mostly the cause of perceptual aliasing however, it can result in virtual agents’ faster learning in early trials. Our free energy method provides a knowledge integration method for the main agent to benefit from that diversity to reduce its regret. It rests upon Thompson sampling policy and behavioral policy of main and virtual agents. Therefore, it is applicable to a variety of tasks, discrete or continuous state space, model-free, and model-based tasks as well as to different reinforcement learning methods. Through a set of experiments, we show that this general framework highly improves learning speed and is clearly superior to previous existing methods. We also provide convergence proof.

强化学习（RL）是生物和人工智能互动学习的核心方法。然而，与人类和动物相比，人工 RL 代理的学习速度非常缓慢，并且受到维度诅咒的影响。部分原因在于孤立地使用 RL，即缺乏社会学习和社会多样性。我们引入了一种基于自由能的社会 RL，用于学习新任务。社会由学习代理和一些不同的虚拟代理组成。这种多样性体现在他们的感知上，而所有代理都使用相同的交互样本进行学习，并共享相同的行动集。感知上的个体差异是造成感知混叠的主要原因，但也可能导致虚拟代理在早期试验中学习速度更快。我们的自由能方法提供了一种知识整合方法，让主代理从这种多样性中获益，从而减少遗憾。它依赖于汤普森采样策略以及主代理和虚拟代理的行为策略。因此，它适用于各种任务、离散或连续状态空间、无模型和基于模型的任务以及不同的强化学习方法。通过一系列实验，我们证明了这种通用框架能极大地提高学习速度，明显优于以往的现有方法。我们还提供了收敛性证明。

{"title":"Learning from different perspectives for regret reduction in reinforcement learning: A free energy approach","authors":"Milad Ghorbani, Reshad Hosseini, Seyed Pooya Shariatpanahi, Majid Nili Ahmadabadi","doi":"10.1016/j.neucom.2024.128797","DOIUrl":"10.1016/j.neucom.2024.128797","url":null,"abstract":"<div><div>Reinforcement learning (RL) is the core method for interactive learning in living and artificial creatures. Nevertheless, in contrast to humans and animals, artificial RL agents are very slow in learning and suffer from the curse of dimensionality. This is partially due to using RL in isolation; i.e. lack of social learning and social diversity. We introduce a free energy-based social RL for learning novel tasks. Society is formed by the learning agent and some diverse virtual ones. That diversity is in their perception while all agents use the same interaction samples for learning and share the same action set. Individual difference in perception is mostly the cause of perceptual aliasing however, it can result in virtual agents’ faster learning in early trials. Our free energy method provides a knowledge integration method for the main agent to benefit from that diversity to reduce its regret. It rests upon Thompson sampling policy and behavioral policy of main and virtual agents. Therefore, it is applicable to a variety of tasks, discrete or continuous state space, model-free, and model-based tasks as well as to different reinforcement learning methods. Through a set of experiments, we show that this general framework highly improves learning speed and is clearly superior to previous existing methods. We also provide convergence proof.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128797"},"PeriodicalIF":5.5,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Diffusion model conditioning on Gaussian mixture model and negative Gaussian mixture gradient 以高斯混合模型和负高斯混合梯度为条件的扩散模型

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128764

Weiguo Lu , Xuan Wu , Deng Ding , Jinqiao Duan , Jirong Zhuang , Gangnan Yuan

Diffusion models (DMs) are a type of generative model that has had a significant impact on image synthesis and beyond. They can incorporate a wide variety of conditioning inputs — such as text or bounding boxes — to guide generation. In this work, we introduce a novel conditioning mechanism that applies Gaussian mixture models (GMMs) for feature conditioning, which helps steer the denoising process in DMs. Drawing on set theory, our comprehensive theoretical analysis reveals that the conditional latent distribution based on features differs markedly from that based on classes. Consequently, feature-based conditioning tends to generate fewer defects than class-based conditioning. Experiments are designed and carried out and the experimental results support our theoretical findings as well as effectiveness of proposed feature conditioning mechanism. Additionally, we propose a new gradient function named the Negative Gaussian Mixture Gradient (NGMG) and incorporate it into the training of diffusion models alongside an auxiliary classifier. We theoretically demonstrate that NGMG offers comparable advantages to the Wasserstein distance, serving as a more effective cost function when learning distributions supported by low-dimensional manifolds, especially in contrast to many likelihood-based cost functions, such as KL divergences.

扩散模型（DM）是生成模型的一种，对图像合成及其他领域产生了重大影响。它们可以结合各种条件输入（如文本或边界框）来指导生成。在这项工作中，我们引入了一种新颖的调节机制，将高斯混合模型（GMM）用于特征调节，从而帮助引导 DM 的去噪过程。借鉴集合论，我们的综合理论分析表明，基于特征的条件潜分布与基于类的条件潜分布有明显不同。因此，与基于类别的调节相比，基于特征的调节往往会产生更少的缺陷。我们设计并进行了实验，实验结果支持了我们的理论发现以及所提出的特征调节机制的有效性。此外，我们还提出了一种名为负高斯混合梯度（Negative Gaussian Mixture Gradient，NGMG）的新梯度函数，并将其与辅助分类器一起纳入扩散模型的训练中。我们从理论上证明了 NGMG 具有与 Wasserstein 距离相当的优势，在学习由低维流形支持的分布时可作为更有效的成本函数，尤其是与 KL 发散等许多基于似然的成本函数相比。

{"title":"Diffusion model conditioning on Gaussian mixture model and negative Gaussian mixture gradient","authors":"Weiguo Lu , Xuan Wu , Deng Ding , Jinqiao Duan , Jirong Zhuang , Gangnan Yuan","doi":"10.1016/j.neucom.2024.128764","DOIUrl":"10.1016/j.neucom.2024.128764","url":null,"abstract":"<div><div>Diffusion models (DMs) are a type of generative model that has had a significant impact on image synthesis and beyond. They can incorporate a wide variety of conditioning inputs — such as text or bounding boxes — to guide generation. In this work, we introduce a novel conditioning mechanism that applies Gaussian mixture models (GMMs) for feature conditioning, which helps steer the denoising process in DMs. Drawing on set theory, our comprehensive theoretical analysis reveals that the conditional latent distribution based on features differs markedly from that based on classes. Consequently, feature-based conditioning tends to generate fewer defects than class-based conditioning. Experiments are designed and carried out and the experimental results support our theoretical findings as well as effectiveness of proposed feature conditioning mechanism. Additionally, we propose a new gradient function named the Negative Gaussian Mixture Gradient (NGMG) and incorporate it into the training of diffusion models alongside an auxiliary classifier. We theoretically demonstrate that NGMG offers comparable advantages to the Wasserstein distance, serving as a more effective cost function when learning distributions supported by low-dimensional manifolds, especially in contrast to many likelihood-based cost functions, such as KL divergences.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128764"},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0