Pub Date : 2024-03-04DOI: 10.1109/TETCI.2024.3360338
Minzhi Chen;Renfang Wang;Yan Qiao;Xin Luo
Dynamic Quality-of-Service (QoS) data can be efficiently represented by a Non-negative Latent-factorization-of-tensors model, which relies on a Non-negative and Multiplicative Update on Incomplete Tensors (NMU-IT) algorithm. Nevertheless, NMU-IT frequently encounters slow convergence and inefficient hyper-parameters selection. Targeting at overcome these critical defects, this paper proposed to improve the NMU-IT algorithm from two perspectives: a) integrating a generalized Nesterov's accelerated gradient method to accelerate the resultant model's convergence rate, and b) establishing the hyper-parameter adaptation mechanism through the particle swarm optimization strategy. On the basis of these conceptions, this study successfully builds a G eneralized Nesterov's Accelerated Gradient-incorporated N on-negative L atent-factorization-of-tensors (GNL) model for precisely and high-efficiently representing the dynamic QoS data. The proposed GNL model has shown its superiority over several advanced models concerning both the precision of estimating missing QoS data and training efficiency, as demonstrated by the experiments conducted on two dynamic QoS datasets.
{"title":"A Generalized Nesterov's Accelerated Gradient-Incorporated Non-Negative Latent-Factorization-of-Tensors Model for Efficient Representation to Dynamic QoS Data","authors":"Minzhi Chen;Renfang Wang;Yan Qiao;Xin Luo","doi":"10.1109/TETCI.2024.3360338","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360338","url":null,"abstract":"Dynamic Quality-of-Service (QoS) data can be efficiently represented by a Non-negative Latent-factorization-of-tensors model, which relies on a Non-negative and Multiplicative Update on Incomplete Tensors (NMU-IT) algorithm. Nevertheless, NMU-IT frequently encounters slow convergence and inefficient hyper-parameters selection. Targeting at overcome these critical defects, this paper proposed to improve the NMU-IT algorithm from two perspectives: a) integrating a generalized Nesterov's accelerated gradient method to accelerate the resultant model's convergence rate, and b) establishing the hyper-parameter adaptation mechanism through the particle swarm optimization strategy. On the basis of these conceptions, this study successfully builds a \u0000<underline>G</u>\u0000eneralized Nesterov's Accelerated Gradient-incorporated \u0000<underline>N</u>\u0000on-negative \u0000<underline>L</u>\u0000atent-factorization-of-tensors (GNL) model for precisely and high-efficiently representing the dynamic QoS data. The proposed GNL model has shown its superiority over several advanced models concerning both the precision of estimating missing QoS data and training efficiency, as demonstrated by the experiments conducted on two dynamic QoS datasets.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-04DOI: 10.1109/TETCI.2024.3367812
Song Gao;Xiaoxuan Wang;Bingbing Song;Renyang Liu;Shaowen Yao;Wei Zhou;Shui Yu
Deep neural networks (DNNs) are sensitive to adversarial examples which are generated by corrupting benign examples with imperceptible perturbations, or have significant changes but can still achieve original prediction results. The latter case is termed as the Type I adversarial example which, however, has limited attention in the literature. In this paper, we introduce two methods, termed HRG and GAG, to generate Type I adversarial examples and attempt to apply them to the privacy-preserving Machine Learning as a Service (MLaaS). Existing methods for the privacy-preserving MLaaS are mostly based on cryptographic techniques, which often incur additional communication and computation overhead, while using Type I adversarial examples to hide users' privacy data is a brand-new exploration. Specifically, HRG utilizes the high-level representations of DNNs to guide generators, and GAG leverages the generative adversarial network to transform original images. Our solution does not involve any model modifications and allows DNNs to run directly on transformed data, thus arousing no additional communication and computation overhead. Extensive experiments on MNIST, CIFAR-10, and ImageNet show that HRG can perfectly hide images into noise and achieve similar accuracy as the original accuracy, and GAG can generate natural images that are completely different from the original images with a small loss of accuracy.
深度神经网络（DNN）对敌意示例非常敏感，敌意示例是通过对良性示例进行难以察觉的扰动而产生的，或者虽然发生了重大变化，但仍能获得原始预测结果。后一种情况被称为 I 型对抗示例，但在文献中的关注度有限。在本文中，我们介绍了两种生成 I 型对抗示例的方法（分别称为 HRG 和 GAG），并尝试将它们应用于保护隐私的机器学习即服务（MLaaS）。现有的保护隐私的机器学习即服务（MLaaS）方法大多基于加密技术，往往会产生额外的通信和计算开销，而利用 I 类对抗示例来隐藏用户的隐私数据则是一种全新的探索。具体来说，HRG 利用 DNN 的高级表示来引导生成器，而 GAG 则利用生成式对抗网络来转换原始图像。我们的解决方案不涉及任何模型修改，允许 DNN 直接在转换后的数据上运行，因此不会产生额外的通信和计算开销。在 MNIST、CIFAR-10 和 ImageNet 上进行的大量实验表明，HRG 可以将图像完美地隐藏到噪声中，并达到与原始精度相似的精度，而 GAG 则可以生成与原始图像完全不同的自然图像，且精度损失很小。
{"title":"Exploiting Type I Adversarial Examples to Hide Data Information: A New Privacy-Preserving Approach","authors":"Song Gao;Xiaoxuan Wang;Bingbing Song;Renyang Liu;Shaowen Yao;Wei Zhou;Shui Yu","doi":"10.1109/TETCI.2024.3367812","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3367812","url":null,"abstract":"Deep neural networks (DNNs) are sensitive to adversarial examples which are generated by corrupting benign examples with imperceptible perturbations, or have significant changes but can still achieve original prediction results. The latter case is termed as the Type I adversarial example which, however, has limited attention in the literature. In this paper, we introduce two methods, termed HRG and GAG, to generate Type I adversarial examples and attempt to apply them to the privacy-preserving Machine Learning as a Service (MLaaS). Existing methods for the privacy-preserving MLaaS are mostly based on cryptographic techniques, which often incur additional communication and computation overhead, while using Type I adversarial examples to hide users' privacy data is a brand-new exploration. Specifically, HRG utilizes the high-level representations of DNNs to guide generators, and GAG leverages the generative adversarial network to transform original images. Our solution does not involve any model modifications and allows DNNs to run directly on transformed data, thus arousing no additional communication and computation overhead. Extensive experiments on MNIST, CIFAR-10, and ImageNet show that HRG can perfectly hide images into noise and achieve similar accuracy as the original accuracy, and GAG can generate natural images that are completely different from the original images with a small loss of accuracy.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evolutionary multitasking optimization (EMTO), owing to its advantage of knowledge sharing, is capable of resolving multiple optimization tasks concurrently. Considering the evolutionary progresses between tasks may be inconsistent, it is necessary for EMTO to regulate the knowledge transfer strategy (KTS), which can alleviate the negative transfer caused by unmatched knowledge. Inspired by this, a multitasking feedback optimization algorithm is proposed with an evolutionary state estimator (MTFO-ESE). First, a multi-source knowledge acquisition strategy (MKA) is introduced to achieve inter-task knowledge, which promotes the tasks to seek the optimization directions in the search space. Second, an evolutionary state estimator (ESE) is established to evaluate the search progress of each task toward the optimal solution. The main idea is to measure the evolutionary pressure of the population under the current individual update strategy using prior and posterior observation. Third, a double-feedback adjustment mechanism (DFBA) is developed to manage KTS based on ESE. This mechanism contributes to alleviating the negative effect caused by unmatched knowledge and eliminating unnecessary exploration. Moreover, the convergence of the proposed MTFO-ESE is analyzed to ensure its effectiveness. Finally, the superior convergence and positive transfer ability of the proposed algorithm are verified through comparative experiments, ablation analyses, and a practical application.
多任务进化优化（EMTO）因其知识共享的优势，能够同时解决多个优化任务。考虑到任务间的进化进度可能不一致，EMTO 有必要调节知识转移策略（KTS），以缓解知识不匹配造成的负转移。受此启发，本文提出了一种带有进化状态估计器的多任务反馈优化算法（MTFO-ESE）。首先，引入多源知识获取策略（MKA）来实现任务间的知识共享，从而促进任务在搜索空间中寻找优化方向。其次，建立了一个进化状态估计器（ESE）来评估每个任务向最优解的搜索进度。其主要思想是利用先验观测和后验观测来衡量当前个体更新策略下种群的进化压力。第三，在 ESE 的基础上开发了一种双重反馈调整机制（DFBA）来管理 KTS。该机制有助于缓解知识不匹配带来的负面影响，消除不必要的探索。此外，还分析了所提出的 MTFO-ESE 的收敛性，以确保其有效性。最后，通过对比实验、消融分析和实际应用，验证了所提算法的卓越收敛性和正迁移能力。
{"title":"Multitasking Feedback Optimization Algorithm Based on an Evolutionary State Estimator","authors":"Xiaolong Wu;Wei Wang;Hongyan Yang;Honggui Han;Junfei Qiao","doi":"10.1109/TETCI.2024.3369314","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3369314","url":null,"abstract":"Evolutionary multitasking optimization (EMTO), owing to its advantage of knowledge sharing, is capable of resolving multiple optimization tasks concurrently. Considering the evolutionary progresses between tasks may be inconsistent, it is necessary for EMTO to regulate the knowledge transfer strategy (KTS), which can alleviate the negative transfer caused by unmatched knowledge. Inspired by this, a multitasking feedback optimization algorithm is proposed with an evolutionary state estimator (MTFO-ESE). First, a multi-source knowledge acquisition strategy (MKA) is introduced to achieve inter-task knowledge, which promotes the tasks to seek the optimization directions in the search space. Second, an evolutionary state estimator (ESE) is established to evaluate the search progress of each task toward the optimal solution. The main idea is to measure the evolutionary pressure of the population under the current individual update strategy using prior and posterior observation. Third, a double-feedback adjustment mechanism (DFBA) is developed to manage KTS based on ESE. This mechanism contributes to alleviating the negative effect caused by unmatched knowledge and eliminating unnecessary exploration. Moreover, the convergence of the proposed MTFO-ESE is analyzed to ensure its effectiveness. Finally, the superior convergence and positive transfer ability of the proposed algorithm are verified through comparative experiments, ablation analyses, and a practical application.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141094884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-28DOI: 10.1109/TETCI.2024.3358103
Pratik K. Mishra;Alex Mihailidis;Shehroz S. Khan
The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacy-protecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them.
{"title":"Skeletal Video Anomaly Detection Using Deep Learning: Survey, Challenges, and Future Directions","authors":"Pratik K. Mishra;Alex Mihailidis;Shehroz S. Khan","doi":"10.1109/TETCI.2024.3358103","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3358103","url":null,"abstract":"The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacy-protecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10453042","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Expensive constrained multi-objective optimization problems (ECMOPs) present a significant challenge to surrogate-assisted evolutionary algorithms (SAEAs) in effectively balancing optimization of the objectives and satisfaction of the constraints with complex landscapes, leading to low feasibility, poor convergence and insufficient diversity. To address these issues, we design a novel algorithm for the automatic selection of two acquisition functions, thereby taking advantage of the benefits of both using and ignoring constraints. Specifically, a multi-objective acquisition function that ignores constraints is proposed to search for problems whose unconstrained Pareto-optimal front (UPF) and constrained Pareto-optimal front (CPF) are similar. In addition, another constrained multi-objective acquisition function is introduced to search for problems whose CPF is far from the UPF. Following the optimization of the two acquisition functions, two model management strategies are proposed to select promising solutions for sampling new solutions and updating the surrogates. Any multi-objective evolutionary algorithm (MOEA) for solving non-constrained and constrained multiobjective optimization problems can be integrated into our algorithm. The performance of the proposed algorithm is evaluated on five suites of test problems, one benchmark-suite of real-world constrained multi-objective optimization problems (RWCMOPs) and a real-world optimization problem. Comparative results show that the proposed algorithm is competitive against state-of-the-art constrained SAEAs.
{"title":"A Surrogate-Assisted Expensive Constrained Multi-Objective Optimization Algorithm Based on Adaptive Switching of Acquisition Functions","authors":"Haofeng Wu;Qingda Chen;Yaochu Jin;Jinliang Ding;Tianyou Chai","doi":"10.1109/TETCI.2024.3359517","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3359517","url":null,"abstract":"Expensive constrained multi-objective optimization problems (ECMOPs) present a significant challenge to surrogate-assisted evolutionary algorithms (SAEAs) in effectively balancing optimization of the objectives and satisfaction of the constraints with complex landscapes, leading to low feasibility, poor convergence and insufficient diversity. To address these issues, we design a novel algorithm for the automatic selection of two acquisition functions, thereby taking advantage of the benefits of both using and ignoring constraints. Specifically, a multi-objective acquisition function that ignores constraints is proposed to search for problems whose unconstrained Pareto-optimal front (UPF) and constrained Pareto-optimal front (CPF) are similar. In addition, another constrained multi-objective acquisition function is introduced to search for problems whose CPF is far from the UPF. Following the optimization of the two acquisition functions, two model management strategies are proposed to select promising solutions for sampling new solutions and updating the surrogates. Any multi-objective evolutionary algorithm (MOEA) for solving non-constrained and constrained multiobjective optimization problems can be integrated into our algorithm. The performance of the proposed algorithm is evaluated on five suites of test problems, one benchmark-suite of real-world constrained multi-objective optimization problems (RWCMOPs) and a real-world optimization problem. Comparative results show that the proposed algorithm is competitive against state-of-the-art constrained SAEAs.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140321738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1109/TETCI.2024.3366771
Xiaoyao Zheng;Xingwang Li;Zhenghua Chen;Liping Sun;Qingying Yu;Liangmin Guo;Yonglong Luo
Compared with traditional recommendation algorithms based on collaborative filtering and content, the sequential recommendation can better capture changes in user interests and recommend items that may be interacted with by the next time according to the user's historical interaction behaviors. Generally, there are several traditional methods for sequential recommendation: Markov Chain (MC) and Deep Neutral Network (DNN), both of which ignore the relationship between various behaviors and the dynamic changes of user interest in items over time. Furthermore, the early research methods usually deal with the user's historical interaction behavior in chronological order, which may cause the loss of partial preference information. According to the perspective that user preferences will change over time, this paper proposes a long and short-term sequential recommendation model with the enhanced self-attention network, RP-SANRec. The short-term intent module of RP-SANRec uses the Gated Recurrent Unit (GRU) to learn the comprehensive historical interaction sequence of the user to calculate the position weight information in the time order, which can be used to enhance the input of the self-attention mechanism. The long-term module captures the user's preferences through a bidirectional long and short-term memory network (Bi-LSTM). Finally, the user's dynamic interests and general preferences are fused, and the following recommendation result is predicted. This article applies the RP-SANRec model to three different public datasets under two evaluation indicators of HR@10 and NDCG@10. The extensive experiments proved that our proposed RP-SANRec model performs better than existing models.
{"title":"Enhanced Self-Attention Mechanism for Long and Short Term Sequential Recommendation Models","authors":"Xiaoyao Zheng;Xingwang Li;Zhenghua Chen;Liping Sun;Qingying Yu;Liangmin Guo;Yonglong Luo","doi":"10.1109/TETCI.2024.3366771","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3366771","url":null,"abstract":"Compared with traditional recommendation algorithms based on collaborative filtering and content, the sequential recommendation can better capture changes in user interests and recommend items that may be interacted with by the next time according to the user's historical interaction behaviors. Generally, there are several traditional methods for sequential recommendation: Markov Chain (MC) and Deep Neutral Network (DNN), both of which ignore the relationship between various behaviors and the dynamic changes of user interest in items over time. Furthermore, the early research methods usually deal with the user's historical interaction behavior in chronological order, which may cause the loss of partial preference information. According to the perspective that user preferences will change over time, this paper proposes a long and short-term sequential recommendation model with the enhanced self-attention network, RP-SANRec. The short-term intent module of RP-SANRec uses the Gated Recurrent Unit (GRU) to learn the comprehensive historical interaction sequence of the user to calculate the position weight information in the time order, which can be used to enhance the input of the self-attention mechanism. The long-term module captures the user's preferences through a bidirectional long and short-term memory network (Bi-LSTM). Finally, the user's dynamic interests and general preferences are fused, and the following recommendation result is predicted. This article applies the RP-SANRec model to three different public datasets under two evaluation indicators of HR@10 and NDCG@10. The extensive experiments proved that our proposed RP-SANRec model performs better than existing models.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096286","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1109/TETCI.2024.3363071
Guanchao Qiao;Ning Ning;Yue Zuo;Pujun Zhou;Mingliang Sun;Shaogang Hu;Qi Yu;Yang Liu
Traditional frame-based cameras capture high-resolution images at specific sampling rates while suffering from motion blur and uneven exposure. Emerging event-based cameras can address these issues with event-driven sampling, but fail to capture texture details. Additional information can be obtained by complementing the characteristics of frame- and event-based sensors. A spatio-temporal fusion spiking neural network (STF-SNN) is proposed here for fusing frame- and event-based information. STF-SNN achieves competitive recognition performance on popular datasets. For example, it achieves 95.77% accuracy on the fusion of CIFAR10 and DVS-CIFAR10, which is 5.01% and 19.27% higher than the non-fused SNN based only on the frame- or event-based information, respectively. To the best of our knowledge, this work first uses SNN to mine spatio-temporal information in the frame-event data stream. The main contributions of this work are: (1) it is proposed to fuse information in the spatio-temporal domain at the feature and decision levels, which yields great accuracy improvement; (2) a weight quantization method for STF-SNN is proposed, which well solves the parameter doubling problem caused by information fusion; (3) it is proposed to prepare data by weak correspondence between frame- and event-based data, which lowers the data preparation barrier of STF-SNN.
{"title":"Spatio-Temporal Fusion Spiking Neural Network for Frame-Based and Event-Based Camera Sensor Fusion","authors":"Guanchao Qiao;Ning Ning;Yue Zuo;Pujun Zhou;Mingliang Sun;Shaogang Hu;Qi Yu;Yang Liu","doi":"10.1109/TETCI.2024.3363071","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3363071","url":null,"abstract":"Traditional frame-based cameras capture high-resolution images at specific sampling rates while suffering from motion blur and uneven exposure. Emerging event-based cameras can address these issues with event-driven sampling, but fail to capture texture details. Additional information can be obtained by complementing the characteristics of frame- and event-based sensors. A spatio-temporal fusion spiking neural network (STF-SNN) is proposed here for fusing frame- and event-based information. STF-SNN achieves competitive recognition performance on popular datasets. For example, it achieves 95.77% accuracy on the fusion of CIFAR10 and DVS-CIFAR10, which is 5.01% and 19.27% higher than the non-fused SNN based only on the frame- or event-based information, respectively. To the best of our knowledge, this work first uses SNN to mine spatio-temporal information in the frame-event data stream. The main contributions of this work are: (1) it is proposed to fuse information in the spatio-temporal domain at the feature and decision levels, which yields great accuracy improvement; (2) a weight quantization method for STF-SNN is proposed, which well solves the parameter doubling problem caused by information fusion; (3) it is proposed to prepare data by weak correspondence between frame- and event-based data, which lowers the data preparation barrier of STF-SNN.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1109/TETCI.2024.3361860
Gengzhi Zhang;Liang Feng;Yu Wang;Min Li;Hong Xie;Kay Chen Tan
To date, transfer learning (TL) has been successfully applied for enhancing the learning performance of reinforcement learning (RL), and many transfer RL (TRL) approaches have been proposed in the literature. However, most of the existing TRL approaches consider knowledge transfer between RL tasks sharing the same state-action space. These methods thus may fail in cases where the RL tasks available for conducting knowledge transfer possess heterogeneous state-action spaces, which is common in many real-world applications. TRL across heterogeneous problem domains is challenging since the differences lie in the state-action spaces of the RL tasks are natural barriers in the knowledge transfer across tasks. This becomes more difficult if multiple heterogeneous source tasks are available when conducting knowledge transfer for a target RL task, as we have to identify the appropriate source task adaptively before performing knowledge transfer towards enhanced RL performance. In this article, we propose a new TRL algorithm with adaptive policy gradient transfer for the cases having multiple heterogeneous source RL tasks. The core ingredients of the proposed algorithm contain a source task selection module to select an appropriate task from a set of heterogeneous source tasks and a knowledge transfer module for conducting knowledge transfer across heterogeneous RL tasks. To investigate the performance of the proposed algorithm, we have conducted comprehensive empirical studies based on the well-known continuous robotic RL task with heterogeneous settings in the number of robot arms (links). The obtained results show that the proposed algorithm is effective and efficient in conducting knowledge transfer across heterogeneous problems for enhanced RL performance, over both the RL algorithm having no knowledge transfer in the learning process and the existing state-of-the-art TRL method.
{"title":"Reinforcement Learning With Adaptive Policy Gradient Transfer Across Heterogeneous Problems","authors":"Gengzhi Zhang;Liang Feng;Yu Wang;Min Li;Hong Xie;Kay Chen Tan","doi":"10.1109/TETCI.2024.3361860","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3361860","url":null,"abstract":"To date, transfer learning (TL) has been successfully applied for enhancing the learning performance of reinforcement learning (RL), and many transfer RL (TRL) approaches have been proposed in the literature. However, most of the existing TRL approaches consider knowledge transfer between RL tasks sharing the same state-action space. These methods thus may fail in cases where the RL tasks available for conducting knowledge transfer possess heterogeneous state-action spaces, which is common in many real-world applications. TRL across heterogeneous problem domains is challenging since the differences lie in the state-action spaces of the RL tasks are natural barriers in the knowledge transfer across tasks. This becomes more difficult if multiple heterogeneous source tasks are available when conducting knowledge transfer for a target RL task, as we have to identify the appropriate source task adaptively before performing knowledge transfer towards enhanced RL performance. In this article, we propose a new TRL algorithm with adaptive policy gradient transfer for the cases having multiple heterogeneous source RL tasks. The core ingredients of the proposed algorithm contain a \u0000<italic>source task selection module</i>\u0000 to select an appropriate task from a set of heterogeneous source tasks and a \u0000<italic>knowledge transfer module</i>\u0000 for conducting knowledge transfer across heterogeneous RL tasks. To investigate the performance of the proposed algorithm, we have conducted comprehensive empirical studies based on the well-known continuous robotic RL task with heterogeneous settings in the number of robot arms (links). The obtained results show that the proposed algorithm is effective and efficient in conducting knowledge transfer across heterogeneous problems for enhanced RL performance, over both the RL algorithm having no knowledge transfer in the learning process and the existing state-of-the-art TRL method.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-20DOI: 10.1109/TETCI.2024.3359038
Zishan Xu;Xiaofeng Zhang;Wei Chen;Jueting Liu;Tingting Xu;Zehua Wang
This paper focuses on the crack detection and digital restoration of ancient mural cultural heritage, proposing a comprehensive method that combines the Unet network structure and diffusion model. Firstly, the Unet network structure is used for efficient crack detection in murals by constructing an ancient mural image dataset for training and validation, achieving accurate identification of mural cracks. Next, an edge-guided optimized masking strategy is adopted for mural restoration, effectively preserving the information of the murals and reducing the damage to the original murals during the restoration process. Lastly, a diffusion model is employed for digital restoration of murals, improving the restoration performance by adjusting parameters to achieve natural repair of mural cracks. Experimental results show that comprehensive method based on the Unet network and diffusion model has significant advantages in the tasks of crack detection and digital restoration of murals, providing a novel and effective approach for the protection and restoration of ancient murals. In addition, this research has significant implications for the technological development in the field of mural restoration and cultural heritage preservation, contributing to the advancement and technological innovation in related fields.
{"title":"MuralDiff: Diffusion for Ancient Murals Restoration on Large-Scale Pre-Training","authors":"Zishan Xu;Xiaofeng Zhang;Wei Chen;Jueting Liu;Tingting Xu;Zehua Wang","doi":"10.1109/TETCI.2024.3359038","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3359038","url":null,"abstract":"This paper focuses on the crack detection and digital restoration of ancient mural cultural heritage, proposing a comprehensive method that combines the Unet network structure and diffusion model. Firstly, the Unet network structure is used for efficient crack detection in murals by constructing an ancient mural image dataset for training and validation, achieving accurate identification of mural cracks. Next, an edge-guided optimized masking strategy is adopted for mural restoration, effectively preserving the information of the murals and reducing the damage to the original murals during the restoration process. Lastly, a diffusion model is employed for digital restoration of murals, improving the restoration performance by adjusting parameters to achieve natural repair of mural cracks. Experimental results show that comprehensive method based on the Unet network and diffusion model has significant advantages in the tasks of crack detection and digital restoration of murals, providing a novel and effective approach for the protection and restoration of ancient murals. In addition, this research has significant implications for the technological development in the field of mural restoration and cultural heritage preservation, contributing to the advancement and technological innovation in related fields.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-19DOI: 10.1109/TETCI.2024.3359046
Zicheng Cai;Lei Chen;Tongtao Ling;Hai-Lin Liu
Differentiable bilevel Neural Architecture Search (NAS) has emerged as a powerful approach in automated machine learning (AutoML) for efficiently searching for neural network architectures. However, the existing differentiable methods encounter challenges, such as the risk of becoming trapped in local optima and the computationally expensive Hessian matrix inverse calculation performed when solving the bilevel NAS optimization model. In this paper, a novel-but-efficient stochastic bilevel optimization approach, called STO-DARTS, is proposed for the bilevel NAS optimization problem. Specifically, we design a hypergradient estimate, which is constructed using stochastic gradient descent from the gradient information contained in the Neumann series. This estimate alleviates the issue of local optima traps, enabling searches for exceptional network architectures. To validate the effectiveness and efficiency of the proposed method, two versions of STO-DARTS with different hypergradient estimators are constructed and experimentally tested on different datasets in NAS-Bench-201 and DARTS search spaces. The experimental results show that the proposed STO-DARTS approach achieves competitive performance with that of other state-of-the-art NAS methods in terms of determining effective network architectures. To support our approach, we also provide theoretical analyses.
可微分双级神经架构搜索（NAS）已成为自动机器学习（AutoML）中高效搜索神经网络架构的有力方法。然而，现有的可微分方法遇到了一些挑战，如陷入局部最优的风险，以及在求解双级 NAS 优化模型时执行计算成本高昂的黑森矩阵逆计算。本文针对双级 NAS 优化问题提出了一种新颖但高效的随机双级优化方法，称为 STO-DARTS。具体来说，我们设计了一种超梯度估计，它是利用随机梯度下降法从诺伊曼数列中包含的梯度信息中构建出来的。这种估计方法缓解了局部最优陷阱的问题，从而能够搜索到特殊的网络架构。为了验证所提方法的有效性和效率，我们在 NAS-Bench-201 和 DARTS 搜索空间的不同数据集上构建了两个版本的带有不同超梯度估计器的 STO-DARTS，并进行了实验测试。实验结果表明，在确定有效的网络架构方面，所提出的 STO-DARTS 方法与其他最先进的 NAS 方法相比，取得了具有竞争力的性能。为了支持我们的方法，我们还提供了理论分析。
{"title":"STO-DARTS: Stochastic Bilevel Optimization for Differentiable Neural Architecture Search","authors":"Zicheng Cai;Lei Chen;Tongtao Ling;Hai-Lin Liu","doi":"10.1109/TETCI.2024.3359046","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3359046","url":null,"abstract":"Differentiable bilevel Neural Architecture Search (NAS) has emerged as a powerful approach in automated machine learning (AutoML) for efficiently searching for neural network architectures. However, the existing differentiable methods encounter challenges, such as the risk of becoming trapped in local optima and the computationally expensive Hessian matrix inverse calculation performed when solving the bilevel NAS optimization model. In this paper, a novel-but-efficient stochastic bilevel optimization approach, called STO-DARTS, is proposed for the bilevel NAS optimization problem. Specifically, we design a hypergradient estimate, which is constructed using stochastic gradient descent from the gradient information contained in the Neumann series. This estimate alleviates the issue of local optima traps, enabling searches for exceptional network architectures. To validate the effectiveness and efficiency of the proposed method, two versions of STO-DARTS with different hypergradient estimators are constructed and experimentally tested on different datasets in NAS-Bench-201 and DARTS search spaces. The experimental results show that the proposed STO-DARTS approach achieves competitive performance with that of other state-of-the-art NAS methods in terms of determining effective network architectures. To support our approach, we also provide theoretical analyses.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141094886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：