Neurocomputing最新文献_第3页

Mitigating gradient conflicts via expert squads in multi-task learning 通过多任务学习中的专家小组缓解梯度冲突

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128832

Jie Chen, Meng Joo Er

The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other state-of-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at https://github.com/chenjie04/Multi-Task-Learning-PyTorch.

多任务学习的基础在于任务之间的协作和互动。然而，在现实世界的众多场景中，某些任务通常需要不同的专业知识。在优化过程中，这些不同任务的特定知识混合在一起往往会导致梯度冲突，这给设计有效的多任务学习系统带来了巨大挑战。本研究提出了一种简单而有效的多任务学习框架，利用专家网络组来分离特定任务知识的学习，并缓解这种梯度冲突。具体来说，这种方法将特征通道分为特定任务和共享部分。特定任务子集由专门的专家处理，以提炼专业知识。共享特征则由来自所有专家的整体输出的点式聚合层捕获，在捕获任务间交互方面表现出卓越的性能。通过同时考虑特定任务知识和共享特征，所提出的方法在多任务学习中表现出了卓越的性能。在 PASCAL-Context 和 NYUD-v2 数据集上进行的大量实验证明，与其他最先进的方法相比，所提出的方法具有更高的性能。此外，还开发了水下场景多任务学习的基准数据集，其中包括物体检测和水下图像增强任务。在该数据集上进行的综合实验一致验证了所提出的多任务学习策略的有效性。源代码见 https://github.com/chenjie04/Multi-Task-Learning-PyTorch。

{"title":"Mitigating gradient conflicts via expert squads in multi-task learning","authors":"Jie Chen, Meng Joo Er","doi":"10.1016/j.neucom.2024.128832","DOIUrl":"10.1016/j.neucom.2024.128832","url":null,"abstract":"<div><div>The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other state-of-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at <span><span>https://github.com/chenjie04/Multi-Task-Learning-PyTorch</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128832"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Secure bipartite consensus of leader–follower multi-agent systems under denial-of-service attacks via observer-based dynamic event-triggered control 通过基于观察者的动态事件触发控制，实现拒绝服务攻击下领导者-追随者多代理系统的安全双方共识

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128817

Haichuan Xu , Fanglai Zhu , Xufeng Ling

This paper investigates secure bipartite consensus (SBC) control problems of leader–follower multi-agent systems (MASs) subjected to denial-of-service (DoS) attacks via observer-based dynamic event-triggered control (DETC). An observer-based DETC protocol with two combining measurements (follower–follower and follower–leader) is first proposed corresponding to valid DoS attack intervals and valid safe intervals. It is concluded that the SBC of MASs without input saturation can be achieved via the observer-based DETC protocol under a signed directed graph if some sufficient inequality conditions hold. And the Zeno behavior can be excluded. Then, a resembling result corresponding to a signed undirected graph is obtained. Furthermore, by using low-gain feedback technic, the semi-global SBC issue under a saturated controller is also considered. Finally, a numerical simulation and two application simulations are displayed to illustrate the effectiveness of the proposed control protocol.

本文通过基于观测者的动态事件触发控制（DETC），研究了遭受拒绝服务（DoS）攻击的领导者-追随者多代理系统（MAS）的安全双向共识（SBC）控制问题。首先提出了一种基于观察者的 DETC 协议，该协议有两个组合测量（追随者-追随者和追随者-领导者），分别对应于有效 DoS 攻击间隔和有效安全间隔。结论是，如果一些充分不等式条件成立，在有符号有向图下可以通过基于观察者的 DETC 协议实现无输入饱和的 MAS SBC。而且可以排除 Zeno 行为。然后，可以得到与有符号无向图相对应的相似结果。此外，通过使用低增益反馈技术，还考虑了饱和控制器下的半全局 SBC 问题。最后，展示了数值模拟和两个应用模拟，以说明所提控制协议的有效性。

引用次数: 0

Interpretable neural network classification model using first-order logic rules 使用一阶逻辑规则的可解释神经网络分类模型

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128840

Haiming Tuo, Zuqiang Meng, Zihao Shi, Daosheng Zhang

Over the past decade, the field of neural networks has made significant strides, particularly in deep learning. However, their limited interpretability has constrained their application in certain critical domains, drawing widespread criticism. Researchers have proposed various methods for explaining neural networks to address this challenge. This paper focuses on rule-based explanations for neural network classification problems. We propose IRCnet, a scalable classification model based on first-order logic rules. IRCnet consists of layers for learning conjunction and disjunction rules, utilizing binary logic activation functions to enhance interpretability. The model is initially trained using a continuous-weight version, which is later binarized to produce a discrete-weight version. During training, we innovatively employed gradient approximation method to handle the non-differentiable weight binarization function, thereby enabling the training of split matrices used for binarization. Finally, Conjunctive Normal Form (CNF) or Disjunctive Normal Form (DNF) rules are extracted from the model’s discrete-weight version. Experimental results indicate that our model achieves the highest or near-highest performance across various classification metrics in multiple structured datasets while demonstrating significant scalability. It effectively balances classification accuracy with the complexity of the generated rules.

过去十年间，神经网络领域取得了长足的进步，尤其是在深度学习方面。然而，神经网络有限的可解释性限制了其在某些关键领域的应用，招致了广泛的批评。为了应对这一挑战，研究人员提出了各种解释神经网络的方法。本文重点关注神经网络分类问题的基于规则的解释。我们提出了基于一阶逻辑规则的可扩展分类模型 IRCnet。IRCnet 由用于学习连接和析取规则的层组成，利用二元逻辑激活函数来增强可解释性。该模型最初使用连续权重版本进行训练，然后将其二值化，生成离散权重版本。在训练过程中，我们创新性地采用了梯度逼近法来处理无差别权重二值化函数，从而实现了用于二值化的分割矩阵的训练。最后，从模型的离散权重版本中提取出连接正则表达式（CNF）或分离正则表达式（DNF）规则。实验结果表明，在多个结构化数据集中，我们的模型在各种分类指标上都取得了最高或接近最高的性能，同时还表现出显著的可扩展性。它有效地平衡了分类准确性和生成规则的复杂性。

{"title":"Interpretable neural network classification model using first-order logic rules","authors":"Haiming Tuo, Zuqiang Meng, Zihao Shi, Daosheng Zhang","doi":"10.1016/j.neucom.2024.128840","DOIUrl":"10.1016/j.neucom.2024.128840","url":null,"abstract":"<div><div>Over the past decade, the field of neural networks has made significant strides, particularly in deep learning. However, their limited interpretability has constrained their application in certain critical domains, drawing widespread criticism. Researchers have proposed various methods for explaining neural networks to address this challenge. This paper focuses on rule-based explanations for neural network classification problems. We propose IRCnet, a scalable classification model based on first-order logic rules. IRCnet consists of layers for learning conjunction and disjunction rules, utilizing binary logic activation functions to enhance interpretability. The model is initially trained using a continuous-weight version, which is later binarized to produce a discrete-weight version. During training, we innovatively employed gradient approximation method to handle the non-differentiable weight binarization function, thereby enabling the training of split matrices used for binarization. Finally, Conjunctive Normal Form (CNF) or Disjunctive Normal Form (DNF) rules are extracted from the model’s discrete-weight version. Experimental results indicate that our model achieves the highest or near-highest performance across various classification metrics in multiple structured datasets while demonstrating significant scalability. It effectively balances classification accuracy with the complexity of the generated rules.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128840"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Event-triggered explorized IRL-based decentralized fault-tolerant guaranteed cost control for interconnected systems against actuator failures 基于事件触发的 IRL 分散容错保证成本控制，用于互联系统，防止执行器故障

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128837

Yuling Liang , Yanhong Luo , Hanguang Su , Xiaoling Zhang , Hongbin Chang , Jun Zhang

This paper presents a novel data-based decentralized guaranteed cost (DGC) fault tolerant control (FTC) scheme for the large-scale systems subject to actuator faults and mismatched interconnection. First, the FTC issues of interconnected systems are converted into a series of near optimal event-triggered control (ETC) methods for isolated subsystems via constructing a modified performance index function of each subsystem. By means of adaptive dynamic programming (ADP) algorithm, the upper bound of performance index function of large-scale systems can be obtained by solving the Hamilton-Jacobi-Bellman (HJB) equation of each auxiliary subsystem. Second, according to the proposed ADP-based decentralized approach and utilizing the event-based synchronous integral reinforcement learning (IRL) algorithm, a model-free guaranteed cost (GC) FTC approach is developed for interconnected large-scale system which can relax the restriction on the condition that system functions must be known. Further, the ultimate uniformly bounded (UUB) stability of auxiliary subsystems can be proved according to the Lyapunov principle. Finally, the effectiveness of the proposed control method is verified by presenting the simulation results.

本文针对存在执行器故障和互联不匹配问题的大型系统，提出了一种新颖的基于数据的分散保证成本（DGC）容错控制（FTC）方案。首先，通过构建每个子系统的修正性能指标函数，将互联系统的 FTC 问题转化为一系列孤立子系统的近优事件触发控制 (ETC) 方法。通过自适应动态编程（ADP）算法，求解各辅助子系统的汉密尔顿-贾可比-贝尔曼（HJB）方程，可获得大规模系统性能指标函数的上界。其次，根据所提出的基于 ADP 的分散方法，并利用基于事件的同步积分强化学习（IRL）算法，为互联大型系统开发了一种无模型保证成本（GC）FTC 方法，该方法可以放宽系统函数必须已知的条件限制。此外，还根据 Lyapunov 原理证明了辅助子系统的终极均匀有界（UUB）稳定性。最后，通过仿真结果验证了所提控制方法的有效性。

{"title":"Event-triggered explorized IRL-based decentralized fault-tolerant guaranteed cost control for interconnected systems against actuator failures","authors":"Yuling Liang , Yanhong Luo , Hanguang Su , Xiaoling Zhang , Hongbin Chang , Jun Zhang","doi":"10.1016/j.neucom.2024.128837","DOIUrl":"10.1016/j.neucom.2024.128837","url":null,"abstract":"<div><div>This paper presents a novel data-based decentralized guaranteed cost (DGC) fault tolerant control (FTC) scheme for the large-scale systems subject to actuator faults and mismatched interconnection. First, the FTC issues of interconnected systems are converted into a series of near optimal event-triggered control (ETC) methods for isolated subsystems via constructing a modified performance index function of each subsystem. By means of adaptive dynamic programming (ADP) algorithm, the upper bound of performance index function of large-scale systems can be obtained by solving the Hamilton-Jacobi-Bellman (HJB) equation of each auxiliary subsystem. Second, according to the proposed ADP-based decentralized approach and utilizing the event-based synchronous integral reinforcement learning (IRL) algorithm, a model-free guaranteed cost (GC) FTC approach is developed for interconnected large-scale system which can relax the restriction on the condition that system functions must be known. Further, the ultimate uniformly bounded (UUB) stability of auxiliary subsystems can be proved according to the Lyapunov principle. Finally, the effectiveness of the proposed control method is verified by presenting the simulation results.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128837"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142652695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observation 利用瞬时观测预测行人过街意图的长短期观测驱动预测网络

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128824

Hui Liu, Chunsheng Liu, Faliang Chang, Yansha Lu, Minhang Liu

Pedestrian crossing intention prediction aims to predict whether the pedestrian will cross the road, which is crucial for the decision-making of intelligent vehicles and ensuring traffic safety. Existing methods just rely on long-term observation and rarely consider it challenging to obtain sufficiently long and precise observation in real-world scenarios. Focus on momentary observation, which only contains two frames of the preceding and current time, we propose a novel Long–Short Observation-driven Prediction Network (LSOP-Net). LSOP-Net comprises two critical components, the Momentary Observation feature Extraction Module (MOE-Module) and the Multimodal Long–Short-term feature Fusion Module (MLSFusion). Utilizing a hybrid training strategy and an external long-term feature pool, the MOE-Module is proposed to extract features with long-term patterns from momentary observations, which effectively mitigates feature deficiency arising from momentary observations. Based on a feature selection fusion mechanism, the MLSFusion is proposed to explicitly model the importance relationship between various modalities’ long–short-term features and the output, which adaptively fuses the long–short-term features from various modalities. Experimental results on the JAAD and PIE datasets demonstrate that our approach achieves superior performance in pedestrian crossing intention prediction with momentary observation.

行人过马路意图预测旨在预测行人是否会过马路，这对智能车辆的决策和确保交通安全至关重要。现有方法仅仅依赖于长期观测，很少考虑在真实世界场景中获得足够长时间和精确观测的挑战性。针对仅包含前一时间和当前时间两帧的瞬间观测，我们提出了一种新型的长短期观测驱动预测网络（LSOP-Net）。LSOP-Net 由两个关键部分组成：瞬间观测特征提取模块（MOE-Module）和多模态长短期特征融合模块（MLSFusion）。MOE 模块利用混合训练策略和外部长期特征库，从瞬时观测中提取具有长期模式的特征，从而有效缓解瞬时观测带来的特征缺陷。在特征选择融合机制的基础上，提出了 MLSFusion，以明确模拟各种模态的长短期特征与输出之间的重要性关系，从而自适应地融合来自各种模态的长短期特征。在 JAAD 和 PIE 数据集上的实验结果表明，我们的方法在利用瞬间观测预测行人过街意图方面取得了优异的性能。

{"title":"Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observation","authors":"Hui Liu, Chunsheng Liu, Faliang Chang, Yansha Lu, Minhang Liu","doi":"10.1016/j.neucom.2024.128824","DOIUrl":"10.1016/j.neucom.2024.128824","url":null,"abstract":"<div><div>Pedestrian crossing intention prediction aims to predict whether the pedestrian will cross the road, which is crucial for the decision-making of intelligent vehicles and ensuring traffic safety. Existing methods just rely on long-term observation and rarely consider it challenging to obtain sufficiently long and precise observation in real-world scenarios. Focus on momentary observation, which only contains two frames of the preceding and current time, we propose a novel <em>Long–Short Observation-driven Prediction Network</em> (LSOP-Net). LSOP-Net comprises two critical components, the <em>Momentary Observation feature Extraction Module</em> (MOE-Module) and the <em>Multimodal Long–Short-term feature Fusion Module</em> (MLSFusion). Utilizing a hybrid training strategy and an external long-term feature pool, the MOE-Module is proposed to extract features with long-term patterns from momentary observations, which effectively mitigates feature deficiency arising from momentary observations. Based on a feature selection fusion mechanism, the MLSFusion is proposed to explicitly model the importance relationship between various modalities’ long–short-term features and the output, which adaptively fuses the long–short-term features from various modalities. Experimental results on the JAAD and PIE datasets demonstrate that our approach achieves superior performance in pedestrian crossing intention prediction with momentary observation.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128824"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure 以句法依存解析结构为指导改进半自回归机器翻译

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128828

Xinran Chen, Sufeng Duan, Gongshen Liu

The advent of non-autoregressive machine translation (NAT) accelerates the decoding superior to autoregressive machine translation (AT) significantly, while bringing about a performance decrease. Semi-autoregressive neural machine translation (SAT), as a compromise, enjoys the merits of both autoregressive and non-autoregressive decoding. However, current SAT methods face the challenges of information-limited initialization and rigorous termination. This paper develops a layer-and-length-based syntactic labeling method and introduces a syntactic dependency parsing structure-guided two-stage semi-autoregressive translation (SDPSAT) structure, which addresses the above challenges with a syntax-based initialization and termination. Additionally, we also present a Mixed Training strategy to shrink exposure bias. Experiments on seven widely-used datasets reveal that our SDPSAT surpasses traditional SAT models with reduced word repetition and achieves competitive results with the AT baseline at a

2 \times \sim 3 \times

speedup.

非自回归机器翻译（NAT）的出现大大加快了优于自回归机器翻译（AT）的解码速度，但同时也带来了性能的下降。半自回归神经机器翻译（SAT）作为一种折中方案，兼具自回归和非自回归解码的优点。然而，目前的 SAT 方法面临着信息有限的初始化和严格终止的挑战。本文开发了一种基于层和长度的句法标注方法，并引入了一种以句法依存解析结构为指导的两阶段半自回归译码（SDPSAT）结构，通过基于句法的初始化和终止解决了上述难题。此外，我们还提出了一种混合训练（Mixed Training）策略，以减少暴露偏差。在七个广泛使用的数据集上进行的实验表明，我们的 SDPSAT 超越了传统的 SAT 模型，减少了单词重复，并以 2×∼3× 的速度取得了与 AT 基线具有竞争力的结果。

{"title":"Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure","authors":"Xinran Chen, Sufeng Duan, Gongshen Liu","doi":"10.1016/j.neucom.2024.128828","DOIUrl":"10.1016/j.neucom.2024.128828","url":null,"abstract":"<div><div>The advent of non-autoregressive machine translation (NAT) accelerates the decoding superior to autoregressive machine translation (AT) significantly, while bringing about a performance decrease. Semi-autoregressive neural machine translation (SAT), as a compromise, enjoys the merits of both autoregressive and non-autoregressive decoding. However, current SAT methods face the challenges of information-limited initialization and rigorous termination. This paper develops a layer-and-length-based syntactic labeling method and introduces a syntactic dependency parsing structure-guided two-stage semi-autoregressive translation (SDPSAT) structure, which addresses the above challenges with a syntax-based initialization and termination. Additionally, we also present a Mixed Training strategy to shrink exposure bias. Experiments on seven widely-used datasets reveal that our SDPSAT surpasses traditional SAT models with reduced word repetition and achieves competitive results with the AT baseline at a <span><math><mrow><mn>2</mn><mo>×</mo><mo>∼</mo><mn>3</mn><mo>×</mo></mrow></math></span> speedup.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128828"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HyV-Summ: Social media video summarization on custom dataset using hybrid techniques HyV-Summ：利用混合技术在定制数据集上进行社交媒体视频摘要

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128852

Jayanta Paul , Anuska Roy , Abhijit Mitra, Jaya Sil

The proliferation of social networking platforms such as YouTube, Facebook, Instagram, and X has led to an exponential growth in multimedia content, with billions of videos uploaded every hour. Efficient management of such vast amount of data necessitates advanced summarization techniques in order to eliminate irrelevant and redundant information. A summarized video, containing the most distinct frames or key frames, provides a concise representation of the original content. Existing deep learning and non-deep learning techniques for video summarization have certain limitations. Deep learning methods are complex and resource-intensive, while non-deep learning algorithms often fail to extract informative features from vast social media videos. This paper addresses the issue by proposing a novel hybrid technique, named Hybrid Video Summarization (HyV-Summ), which integrates deep and non-deep learning techniques to leverage their respective strengths by focusing only on social media content. We developed a custom dataset, SocialSum to train our proposed model HyV-Summ, since existing benchmark datasets like TVSum and SumMe contain diverse types of content not specific to social media videos. We provide a comparative analysis of existing techniques and datasets with our proposed techniques and dataset. The results demonstrate that HyV-Summ outperforms existing techniques, such as Long Short Term Memory (LSTM)-based and Generative Adversarial Network (GAN)-based summarization by achieving higher F1-scores while applied on both the SocialSum dataset and available datasets.

随着 YouTube、Facebook、Instagram 和 X 等社交网络平台的普及，多媒体内容呈指数级增长，每小时上传的视频数量高达数十亿。要有效管理如此海量的数据，就必须采用先进的摘要技术，以消除不相关的冗余信息。概括后的视频包含最独特的帧或关键帧，能够简明扼要地呈现原始内容。现有的用于视频摘要的深度学习和非深度学习技术都有一定的局限性。深度学习方法复杂且资源密集，而非深度学习算法往往无法从海量社交媒体视频中提取信息特征。为了解决这个问题，本文提出了一种新颖的混合技术，名为混合视频摘要（HyV-Summ），它整合了深度学习和非深度学习技术，只关注社交媒体内容，从而发挥了它们各自的优势。由于 TVSum 和 SumMe 等现有基准数据集包含各种类型的内容，而非社交媒体视频的特定内容，因此我们开发了一个定制数据集 SocialSum 来训练我们提出的模型 HyV-Summ。我们将现有技术和数据集与我们提出的技术和数据集进行了对比分析。结果表明，HyV-Summ 的性能优于现有技术，如基于长短期记忆（LSTM）和基于生成对抗网络（GAN）的摘要技术，在应用于 SocialSum 数据集和现有数据集时都能获得更高的 F1 分数。

{"title":"HyV-Summ: Social media video summarization on custom dataset using hybrid techniques","authors":"Jayanta Paul , Anuska Roy , Abhijit Mitra, Jaya Sil","doi":"10.1016/j.neucom.2024.128852","DOIUrl":"10.1016/j.neucom.2024.128852","url":null,"abstract":"<div><div>The proliferation of social networking platforms such as YouTube, Facebook, Instagram, and X has led to an exponential growth in multimedia content, with billions of videos uploaded every hour. Efficient management of such vast amount of data necessitates advanced summarization techniques in order to eliminate irrelevant and redundant information. A summarized video, containing the most distinct frames or key frames, provides a concise representation of the original content. Existing deep learning and non-deep learning techniques for video summarization have certain limitations. Deep learning methods are complex and resource-intensive, while non-deep learning algorithms often fail to extract informative features from vast social media videos. This paper addresses the issue by proposing a novel hybrid technique, named Hybrid Video Summarization (<strong>HyV-Summ</strong>), which integrates deep and non-deep learning techniques to leverage their respective strengths by focusing only on social media content. We developed a custom dataset, <strong>SocialSum</strong> to train our proposed model <strong>HyV-Summ</strong>, since existing benchmark datasets like TVSum and SumMe contain diverse types of content not specific to social media videos. We provide a comparative analysis of existing techniques and datasets with our proposed techniques and dataset. The results demonstrate that HyV-Summ outperforms existing techniques, such as Long Short Term Memory (LSTM)-based and Generative Adversarial Network (GAN)-based summarization by achieving higher F1-scores while applied on both the SocialSum dataset and available datasets.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128852"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Deep feature response discriminative calibration 深度特征响应判别校准

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128848

Wenxiang Xu , Tian Qiu , Linyun Zhou , Zunlei Feng , Mingli Song , Huiqiong Wang

Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach.

深度神经网络（DNN）在各个领域都有大量应用。为了提高模型的准确性，人们提出了一些优化技术，如 ResNet 和 SENet。这些技术根据统一标准调整或校准特征响应，从而提高模型性能。然而，它们缺乏对不同特征的判别校准，从而给模型输出带来了局限性。因此，我们提出了一种对特征响应进行判别校准的方法。初步实验结果表明，神经特征响应遵循高斯分布。因此，我们利用高斯概率密度函数计算置信度值，然后将这些值与原始响应值进行整合。这种整合的目的是提高神经特征响应的特征判别能力。在校准值的基础上，我们提出了一种基于插件的校准模块，并将其集成到改进的 ResNet 架构中，称为响应校准网络（ResCNet）。在 CIFAR-10、CIFAR-100、SVHN 和 ImageNet 等数据集上进行的广泛实验证明了所提方法的有效性。

{"title":"Deep feature response discriminative calibration","authors":"Wenxiang Xu , Tian Qiu , Linyun Zhou , Zunlei Feng , Mingli Song , Huiqiong Wang","doi":"10.1016/j.neucom.2024.128848","DOIUrl":"10.1016/j.neucom.2024.128848","url":null,"abstract":"<div><div>Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128848"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142652693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-modal soft prompt-tuning for Chinese Clickbait Detection 针对中文点击诱饵检测的多模式软提示调整

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128829

Ye Wang , Yi Zhu , Yun Li , Liting Wei , Yunhao Yuan , Jipeng Qiang

With the rapid growth of Chinese online services, clickbait has proliferated at an unprecedented rate, designed to manipulate users into clicking for increased traffic or advertising promotion. Such clickbait not only facilitates the spread of fake news and misinformation but also enables click-jacking attacks, redirecting users to deceptive websites that steal personal information. These harmful activities can result in significant losses and serious repercussions. The widespread presence of clickbait underscores both the importance and the challenges of developing effective detection methods. To date, the research paradigm of clickbait detection evolved from deep neural networks to fine-tuned Pre-trained Language Models (PLMs) and, more recently, into prompt-tuning models. However, these methods may suffer two main limitations: (1) they fail to utilize the multi-modal context information in news or posts and explore the higher-level feature representations to enhance the performance of clickbait detection; (2) they largely ignore the diverse range of Chinese expressive forms and neglect the complex semantics and syntactic structures of textual content to assist in learning a better news representation. To overcome these limitations, we proposed a Multi-modal Soft Prompt-tuning Method (MSP) for Chinese Clickbait Detection, which jointly models the textual and image information into a continuous prompt embedding as the input of PLMs. Specifically, firstly, the soft prompt-tuning model including Graph Attention Network and Contrastive Language-Image Pre-training are employed to learn the feature representations of texts and images in news or posts, respectively. Then the obtained text and image representations are re-input into the soft prompt-tuning model with automatic template generation. The extensive experiments on three Chinese clickbait detection datasets demonstrate that our MSP achieved state-of-the-art performance.

随着中国在线服务的快速发展，点击诱饵以前所未有的速度激增，其目的是操纵用户点击以增加流量或进行广告推广。此类点击诱饵不仅助长了虚假新闻和错误信息的传播，还促成了点击劫持攻击，将用户重定向到窃取个人信息的欺骗性网站。这些有害活动会造成重大损失和严重影响。点击诱饵的广泛存在凸显了开发有效检测方法的重要性和挑战性。迄今为止，点击诱饵检测的研究范式已从深度神经网络发展到微调预训练语言模型（PLMs），最近又发展到提示微调模型。然而，这些方法可能存在两个主要局限：（1）它们未能利用新闻或帖子中的多模态上下文信息，探索更高层次的特征表征，以提高点击诱饵检测的性能；（2）它们在很大程度上忽视了中文表达形式的多样性，忽略了文本内容中复杂的语义和句法结构对学习更好的新闻表征的帮助。为了克服这些局限性，我们提出了一种用于中文点击诱饵检测的多模态软提示调整方法（MSP），该方法将文本和图像信息联合建模为连续的提示嵌入，作为 PLM 的输入。具体来说，首先，利用包括图形注意网络和对比语言-图像预训练在内的软提示调整模型，分别学习新闻或帖子中的文本和图像特征表征。然后将获得的文本和图像表征重新输入软提示调谐模型，并自动生成模板。在三个中文点击诱饵检测数据集上的大量实验证明，我们的 MSP 达到了最先进的性能。

{"title":"Multi-modal soft prompt-tuning for Chinese Clickbait Detection","authors":"Ye Wang , Yi Zhu , Yun Li , Liting Wei , Yunhao Yuan , Jipeng Qiang","doi":"10.1016/j.neucom.2024.128829","DOIUrl":"10.1016/j.neucom.2024.128829","url":null,"abstract":"<div><div>With the rapid growth of Chinese online services, clickbait has proliferated at an unprecedented rate, designed to manipulate users into clicking for increased traffic or advertising promotion. Such clickbait not only facilitates the spread of fake news and misinformation but also enables click-jacking attacks, redirecting users to deceptive websites that steal personal information. These harmful activities can result in significant losses and serious repercussions. The widespread presence of clickbait underscores both the importance and the challenges of developing effective detection methods. To date, the research paradigm of clickbait detection evolved from deep neural networks to fine-tuned Pre-trained Language Models (PLMs) and, more recently, into prompt-tuning models. However, these methods may suffer two main limitations: (1) they fail to utilize the multi-modal context information in news or posts and explore the higher-level feature representations to enhance the performance of clickbait detection; (2) they largely ignore the diverse range of Chinese expressive forms and neglect the complex semantics and syntactic structures of textual content to assist in learning a better news representation. To overcome these limitations, we proposed a Multi-modal Soft Prompt-tuning Method (MSP) for Chinese Clickbait Detection, which jointly models the textual and image information into a continuous prompt embedding as the input of PLMs. Specifically, firstly, the soft prompt-tuning model including Graph Attention Network and Contrastive Language-Image Pre-training are employed to learn the feature representations of texts and images in news or posts, respectively. Then the obtained text and image representations are re-input into the soft prompt-tuning model with automatic template generation. The extensive experiments on three Chinese clickbait detection datasets demonstrate that our MSP achieved state-of-the-art performance.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128829"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Quantifying neural network uncertainty under volatility clustering 量化波动聚类下的神经网络不确定性

IF 5.5 2区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Neurocomputing

Pub Date : 2024-11-07 DOI: 10.1016/j.neucom.2024.128816

Steven Y.K. Wong , Jennifer S.K. Chan , Lamiae Azizi

Time-series with volatility clustering pose a unique challenge to uncertainty quantification (UQ) for returns forecasts. Methods for UQ such as Deep Evidential regression offer a simple way of quantifying return forecast uncertainty without the costs of a full Bayesian treatment. However, the Normal-Inverse-Gamma (NIG) prior adopted by Deep Evidential regression is prone to miscalibration as the NIG prior is assigned to latent mean and variance parameters in a hierarchical structure. Moreover, it also overparameterizes the marginal data distribution. These limitations may affect the accurate delineation of epistemic (model) and aleatoric (data) uncertainties. We propose a Scale Mixture Distribution as a simpler alternative which can provide favourable complexity-accuracy trade-off and assign separate subnetworks to each model parameter. To illustrate the performance of our proposed method, we apply it to two sets of financial time-series exhibiting volatility clustering: cryptocurrencies and U.S. equities and test the performance in some ablation studies.

具有波动集群的时间序列对收益预测的不确定性量化（UQ）提出了独特的挑战。深度求证回归等不确定性量化方法提供了一种量化回报预测不确定性的简单方法，而无需付出完全贝叶斯处理方法的成本。然而，深度求证回归所采用的正态逆伽马（NIG）先验容易出现误判，因为 NIG 先验是以分层结构分配给潜在均值和方差参数的。此外，它还会对边际数据分布进行过度参数化。这些局限性可能会影响对认识（模型）和估计（数据）不确定性的准确划分。我们提出了规模混合分布作为一种更简单的替代方法，它可以在复杂性和准确性之间做出有利的权衡，并为每个模型参数分配单独的子网络。为了说明我们提出的方法的性能，我们将其应用于两组表现出波动性聚类的金融时间序列：加密货币和美国股票，并在一些消融研究中测试其性能。

引用次数: 0