Pub Date : 2024-02-19DOI: 10.1109/TETCI.2024.3360308
Zhidong Wang;He Huang
This paper focuses on proposing a second-order complex-valued incremental learning (CVIL) algorithm for the structure optimization of fully complex-valued neural networks (FCVNNs). The main purpose of this study is to integrate the structure optimization and parameter learning of FCVNNs into a unified framework such that good generalization is guaranteed. A hybrid training strategy is firstly developed for FCVNNs with fixed structure. By introducing complex-valued sparse matrices and generalized augmented hidden output matrix, nonlinear parameters between the hidden and input neurons are trained by complex-valued Levenberg-Marquardt (CLM) algorithm and linear parameters between the output and hidden neurons are obtained by complex-valued least squares (CLS) algorithm. Starting with an initial FCVNN, hidden neurons are added one by one once the training falls in the plateau. It is theoretically shown that the objective function is monotonously decreasing after adding hidden neuron and successive learning is immediately continuous with the latest training results. Repetition training is avoided and thus training efficiency is achieved. The experimental results on the channel modulation identification and real-valued pattern classification tasks are provided to demonstrate that the developed algorithm is superior to some existing ones for the training of FCVNNs.
{"title":"Second-Order Structure Optimization of Fully Complex-Valued Neural Networks","authors":"Zhidong Wang;He Huang","doi":"10.1109/TETCI.2024.3360308","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360308","url":null,"abstract":"This paper focuses on proposing a second-order complex-valued incremental learning (CVIL) algorithm for the structure optimization of fully complex-valued neural networks (FCVNNs). The main purpose of this study is to integrate the structure optimization and parameter learning of FCVNNs into a unified framework such that good generalization is guaranteed. A hybrid training strategy is firstly developed for FCVNNs with fixed structure. By introducing complex-valued sparse matrices and generalized augmented hidden output matrix, nonlinear parameters between the hidden and input neurons are trained by complex-valued Levenberg-Marquardt (CLM) algorithm and linear parameters between the output and hidden neurons are obtained by complex-valued least squares (CLS) algorithm. Starting with an initial FCVNN, hidden neurons are added one by one once the training falls in the plateau. It is theoretically shown that the objective function is monotonously decreasing after adding hidden neuron and successive learning is immediately continuous with the latest training results. Repetition training is avoided and thus training efficiency is achieved. The experimental results on the channel modulation identification and real-valued pattern classification tasks are provided to demonstrate that the developed algorithm is superior to some existing ones for the training of FCVNNs.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-16DOI: 10.1109/TETCI.2024.3360290
Yihan Wang;Yongfang Wang;Tengyao Cui;Zhijun Fang
Video-based Point Cloud Compression (V-PCC) was proposed by the Moving Picture Experts Group (MPEG) to standardize Point Cloud Compression (PCC). The main idea of V-PCC is to project the Dynamic Point Cloud (DPC) into auxiliary information, occupancy, geometry, and attribute videos for encoding utilizing High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), etc. Compared with the previous PCC algorithms, V-PCC has achieved a significant improvement in compression efficiency. However, it is accompanied by substantial computational complexity. To solve this problem, this paper proposes a fast V-PCC method to decrease the coding complexity. Taking into account the coding characteristic of V-PCC, the geometry and attribute maps are first classified into occupied and unoccupied blocks. Moreover, we analyze Coding Unit (CU) splitting for geometry and attribute map. Finally, we propose fast V-PCC algorithms based on early termination algorithm and transformer model, in which the early termination method is proposed for low complexity blocks in the geometry and attribute map, and the transformer model-based fast method is designed to predict the optimal CU splitting modes for the occupied block of the attribute map. The proposed algorithms are implemented with typical DPC sequences on the Test Model Category 2 (TMC2). The experimental results imply that the average time of the proposed method can significantly reduce 56.39% and 55.10% in the geometry and attribute map, respectively, with negligible Bjontegaard-Delta bitrate (BD-rate) compared with the anchor method.
{"title":"Fast Video-Based Point Cloud Compression Based on Early Termination and Transformer Model","authors":"Yihan Wang;Yongfang Wang;Tengyao Cui;Zhijun Fang","doi":"10.1109/TETCI.2024.3360290","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360290","url":null,"abstract":"Video-based Point Cloud Compression (V-PCC) was proposed by the Moving Picture Experts Group (MPEG) to standardize Point Cloud Compression (PCC). The main idea of V-PCC is to project the Dynamic Point Cloud (DPC) into auxiliary information, occupancy, geometry, and attribute videos for encoding utilizing High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), etc. Compared with the previous PCC algorithms, V-PCC has achieved a significant improvement in compression efficiency. However, it is accompanied by substantial computational complexity. To solve this problem, this paper proposes a fast V-PCC method to decrease the coding complexity. Taking into account the coding characteristic of V-PCC, the geometry and attribute maps are first classified into occupied and unoccupied blocks. Moreover, we analyze Coding Unit (CU) splitting for geometry and attribute map. Finally, we propose fast V-PCC algorithms based on early termination algorithm and transformer model, in which the early termination method is proposed for low complexity blocks in the geometry and attribute map, and the transformer model-based fast method is designed to predict the optimal CU splitting modes for the occupied block of the attribute map. The proposed algorithms are implemented with typical DPC sequences on the Test Model Category 2 (TMC2). The experimental results imply that the average time of the proposed method can significantly reduce 56.39% and 55.10% in the geometry and attribute map, respectively, with negligible Bjontegaard-Delta bitrate (BD-rate) compared with the anchor method.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper investigates the application of multi-agent reinforcement learning (MARL) algorithm to solve the joint spectrum and power allocation problem (JSPAP) in wireless network. The objective of JSPAP is to optimize the subband selection and transmit power levels for links, with the aim of maximizing the sum-rate utility function. To address the JSPAP with discrete subband selection and continuous power allocation, most existing algorithms rely on a centralized optimizer and the instantaneous global channel state information, which can be challenging to implement in large wireless networks with time-varying subbands. To conquer such limitation, a two-stage MARL algorithm is proposed, which comprises a top layer network for selecting subbands across all links and a bottom layer network for determining the transmit power levels for all transmitters. By utilizing the value decomposition technique in the top layer network, the links can cooperatively select transmission subbands, effectively resolving non-stationarity issues in wireless network. Additionally, in the bottom layer network of the proposed two-stage MARL algorithm, each transmitter selects the transmit power level based solely on the local information, thereby effectively reducing computational burden. Empirical experiments demonstrate the effectiveness of the proposed two-stage MARL algorithm by comparison with the state-of-the-art RL algorithms and fractional programming algorithms.
{"title":"Joint Spectrum and Power Allocation in Wireless Network: A Two-Stage Multi-Agent Reinforcement Learning Method","authors":"Pengcheng Dai;He Wang;Huazhou Hou;Xusheng Qian;Wenwu Yu","doi":"10.1109/TETCI.2024.3360305","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360305","url":null,"abstract":"This paper investigates the application of multi-agent reinforcement learning (MARL) algorithm to solve the joint spectrum and power allocation problem (JSPAP) in wireless network. The objective of JSPAP is to optimize the subband selection and transmit power levels for links, with the aim of maximizing the sum-rate utility function. To address the JSPAP with discrete subband selection and continuous power allocation, most existing algorithms rely on a centralized optimizer and the instantaneous global channel state information, which can be challenging to implement in large wireless networks with time-varying subbands. To conquer such limitation, a two-stage MARL algorithm is proposed, which comprises a top layer network for selecting subbands across all links and a bottom layer network for determining the transmit power levels for all transmitters. By utilizing the value decomposition technique in the top layer network, the links can cooperatively select transmission subbands, effectively resolving non-stationarity issues in wireless network. Additionally, in the bottom layer network of the proposed two-stage MARL algorithm, each transmitter selects the transmit power level based solely on the local information, thereby effectively reducing computational burden. Empirical experiments demonstrate the effectiveness of the proposed two-stage MARL algorithm by comparison with the state-of-the-art RL algorithms and fractional programming algorithms.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.
{"title":"Autopilot Controller of Fixed-Wing Planes Based on Curriculum Reinforcement Learning Scheduled by Adaptive Learning Curve","authors":"Lun Li;Xuebo Zhang;Chenxu Qian;Runhua Wang;Minghui Zhao","doi":"10.1109/TETCI.2024.3360322","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360322","url":null,"abstract":"In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141096290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-16DOI: 10.1109/TETCI.2024.3360316
Abhishek Moitra;Abhiroop Bhattacharjee;Youngeun Kim;Priyadarshini Panda
In practical cloud-edge scenarios, where a resource constrained edge performs data acquisition and a cloud system (having sufficient resources) performs inference tasks with a deep neural network (DNN), adversarial robustness is critical for reliability and ubiquitous deployment. Adversarial detection is a prime adversarial defense technique used in prior literature. However, in prior detection works, the detector is attached to the classifier model and both detector and classifier work in tandem to perform adversarial detection that requires a high computational overhead which is not available at the lowpower edge. Therefore, prior works can only perform adversarial detection at the cloud and not at the edge. This means that in case of adversarial attacks, the unfavourable adversarial samples must be communicated to the cloud which leads to energy wastage at the edge device. Therefore, a low-power edge-friendly adversarial detection method is required to improve the energy efficiency of the edge and robustness of the cloud-based classifier. To this end, RobustEdge proposes Quantization-enabled Energy Separation (QES) training with “early detection and exit” to perform edge-based low cost adversarial detection. The QEStrained detector implemented at the edge blocks adversarial data transmission to the classifier model, thereby improving adversarial robustness and energy-efficiency of the Cloud-Edge system. Through extensive experiments on CIFAR10, CIFAR100 and TinyImagenet, we find that 16-bit and 12-bit quantized detectors achieve a high AUC score $>$ 0.9 while improving the energy-efficiency of the cloud-edge system by $>166times$ compared to prior cloud-based adversarial detection approaches.
{"title":"RobustEdge: Low Power Adversarial Detection for Cloud-Edge Systems","authors":"Abhishek Moitra;Abhiroop Bhattacharjee;Youngeun Kim;Priyadarshini Panda","doi":"10.1109/TETCI.2024.3360316","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360316","url":null,"abstract":"In practical cloud-edge scenarios, where a resource constrained edge performs data acquisition and a cloud system (having sufficient resources) performs inference tasks with a deep neural network (DNN), adversarial robustness is critical for reliability and ubiquitous deployment. Adversarial detection is a prime adversarial defense technique used in prior literature. However, in prior detection works, the detector is attached to the classifier model and both detector and classifier work in tandem to perform adversarial detection that requires a high computational overhead which is not available at the lowpower edge. Therefore, prior works can only perform adversarial detection at the cloud and not at the edge. This means that in case of adversarial attacks, the unfavourable adversarial samples must be communicated to the cloud which leads to energy wastage at the edge device. Therefore, a low-power edge-friendly adversarial detection method is required to improve the energy efficiency of the edge and robustness of the cloud-based classifier. To this end, RobustEdge proposes Quantization-enabled Energy Separation (QES) training with “early detection and exit” to perform edge-based low cost adversarial detection. The QEStrained detector implemented at the edge blocks adversarial data transmission to the classifier model, thereby improving adversarial robustness and energy-efficiency of the Cloud-Edge system. Through extensive experiments on CIFAR10, CIFAR100 and TinyImagenet, we find that 16-bit and 12-bit quantized detectors achieve a high AUC score \u0000<inline-formula><tex-math>$>$</tex-math></inline-formula>\u0000 0.9 while improving the energy-efficiency of the cloud-edge system by \u0000<inline-formula><tex-math>$>166times$</tex-math></inline-formula>\u0000 compared to prior cloud-based adversarial detection approaches.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140321736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-15DOI: 10.1109/TETCI.2024.3360329
Tuo Feng;Dongbing Gu
Depth map and ego-motion estimations from monocular consecutive images are challenging to unsupervised learning Visual Odometry (VO) approaches. This paper proposes a novel VO architecture: Recurrent Spatial-Temporal Network (RSTNet), which can estimate the depth map and ego-motion from monocular consecutive images. The main contributions in this paper include a novel RST-encoder layer and its corresponding RST-decoder layer, which can preserve and recover spatial and temporal features from inputs. Our RSTNet extracts appearance features from input images, and extracts structure and temporal features from intermediate results for ego-motion estimation. Our RSTNet also includes a pre-trained network to detect dynamic objects from the difference between full and rigid optical flows. A novel auto-mask scheme is designed in the loss function to deal with some challenging scenes. Our evaluation results on the KITTI odometry benchmark show our RSTNet outperforms some of the existing unsupervised learning approaches.
{"title":"RSTNet: Recurrent Spatial-Temporal Networks for Estimating Depth and Ego-Motion","authors":"Tuo Feng;Dongbing Gu","doi":"10.1109/TETCI.2024.3360329","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360329","url":null,"abstract":"Depth map and ego-motion estimations from monocular consecutive images are challenging to unsupervised learning Visual Odometry (VO) approaches. This paper proposes a novel VO architecture: Recurrent Spatial-Temporal Network (RSTNet), which can estimate the depth map and ego-motion from monocular consecutive images. The main contributions in this paper include a novel RST-encoder layer and its corresponding RST-decoder layer, which can preserve and recover spatial and temporal features from inputs. Our RSTNet extracts appearance features from input images, and extracts structure and temporal features from intermediate results for ego-motion estimation. Our RSTNet also includes a pre-trained network to detect dynamic objects from the difference between full and rigid optical flows. A novel auto-mask scheme is designed in the loss function to deal with some challenging scenes. Our evaluation results on the KITTI odometry benchmark show our RSTNet outperforms some of the existing unsupervised learning approaches.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141094883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-13DOI: 10.1109/TETCI.2024.3359093
Lei Zhang;Huabin Zhang;Zihao Chen;Sibo Liu;Haipeng Yang;Hongke Zhao
Multi-objective evolutionary algorithms (MOEAs) have been proved to be competitive in recommender systems. As the application scenarios of recommender systems become increasingly complex, the number of objectives to be considered in the recommender systems increases. However, most existing multi-objective recommendation algorithms lead to increased environmental selection pressure as the number of objectives increases. To tackle the issue, in this paper, we propose a multi-population based evolutionary algorithm named MP-MORS for many-objective recommendations, where two subpopulations and one major population are used to evolve and interact to find high-quality solutions. Specifically, the objectives are firstly divided into those evaluated on individual users (defined as IndObjectives) and those evaluated on all users (defined as as AllObjectives). Then two subpopulations are suggested to optimize the two types of objectives respectively, with which the potential good solutions can be easily found. In addition, the major population considers the balance of all objectives and refines these potential good solutions. Finally, a set of high-quality solutions can be obtained by the proposed adaptive population interaction strategy. Experiments on the datasets Movielens and Douban show that the proposed MP-MORS outperforms the state-of-the-art algorithms for many-objective recommendations.
{"title":"A Multi-Population Based Evolutionary Algorithm for Many-Objective Recommendations","authors":"Lei Zhang;Huabin Zhang;Zihao Chen;Sibo Liu;Haipeng Yang;Hongke Zhao","doi":"10.1109/TETCI.2024.3359093","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3359093","url":null,"abstract":"Multi-objective evolutionary algorithms (MOEAs) have been proved to be competitive in recommender systems. As the application scenarios of recommender systems become increasingly complex, the number of objectives to be considered in the recommender systems increases. However, most existing multi-objective recommendation algorithms lead to increased environmental selection pressure as the number of objectives increases. To tackle the issue, in this paper, we propose a multi-population based evolutionary algorithm named MP-MORS for many-objective recommendations, where two subpopulations and one major population are used to evolve and interact to find high-quality solutions. Specifically, the objectives are firstly divided into those evaluated on individual users (defined as IndObjectives) and those evaluated on all users (defined as as AllObjectives). Then two subpopulations are suggested to optimize the two types of objectives respectively, with which the potential good solutions can be easily found. In addition, the major population considers the balance of all objectives and refines these potential good solutions. Finally, a set of high-quality solutions can be obtained by the proposed adaptive population interaction strategy. Experiments on the datasets \u0000<italic>Movielens</i>\u0000 and \u0000<italic>Douban</i>\u0000 show that the proposed MP-MORS outperforms the state-of-the-art algorithms for many-objective recommendations.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140310162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-13DOI: 10.1109/TETCI.2024.3360282
Da Liu;Qun Zong;Xiuyun Zhang;Ruilong Zhang;Liqian Dou;Bailing Tian
Due to the characteristics of the small size and low cost of unmanned aerial vehicles (UAVs), Multi-UAV confrontation will play an important role in future wars. The Multi-UAV confrontation game in the air combat environment is investigated in this paper. To truly deduce the confrontation scene, a physics engine is established based on the Multi-UAV Confrontation Scenario (MCS) framework, enabling the real-time interaction between the agent and environment while making the learned strategies more realistic. To form an effective confrontation strategy, the Graph Attention Multi-agent Soft Actor Critic Reinforcement Learning with Target Predicting Network (GA-MASAC-TP Net) is firstly proposed for Multi-UAV confrontation game. The merits lie in that the Multi-UAV trajectory prediction, considering interactions among targets, is incorporated innovatively into the Multi-agent reinforcement learning (MARL), enabling Multi-UAVs to make decisions more accurately based on situation prediction. Specifically, the Soft Actor Critic (SAC) algorithm is extended to the Multi-agent domain and embed with the graph attention neural network into the Actor, Critic network, so the UAV could aggregate the information of the spatial neighbor teammates based on the attention mechanism for better collaboration. The comparative experiment and ablation study demonstrate the effectiveness of the proposed algorithm and the state-of-art performance in the MCS.
由于无人机体积小、成本低的特点，多无人机对抗将在未来战争中扮演重要角色。本文研究了空战环境下的多无人机对抗博弈。为了真实推演对抗场景，本文建立了基于多无人机对抗场景（MCS）框架的物理引擎，实现了代理与环境的实时交互，同时使学习到的策略更加逼真。为了形成有效的对抗策略，首先提出了针对多无人机对抗博弈的图注意多代理软代理批评强化学习与目标预测网络（GA-MASAC-TP Net）。其优点在于创新性地将考虑目标间相互作用的多无人机轨迹预测纳入多代理强化学习（MARL），使多无人机能够根据情况预测做出更准确的决策。具体而言，将软行动者批判（Soft Actor Critic，SAC）算法扩展到多机器人领域，并将图注意神经网络嵌入到行动者、批判网络中，使无人机可以基于注意机制聚合空间相邻队友的信息，以实现更好的协作。对比实验和消融研究证明了所提算法的有效性，以及在 MCS 中的先进性能。
{"title":"Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation","authors":"Da Liu;Qun Zong;Xiuyun Zhang;Ruilong Zhang;Liqian Dou;Bailing Tian","doi":"10.1109/TETCI.2024.3360282","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360282","url":null,"abstract":"Due to the characteristics of the small size and low cost of unmanned aerial vehicles (UAVs), Multi-UAV confrontation will play an important role in future wars. The Multi-UAV confrontation game in the air combat environment is investigated in this paper. To truly deduce the confrontation scene, a physics engine is established based on the Multi-UAV Confrontation Scenario (MCS) framework, enabling the real-time interaction between the agent and environment while making the learned strategies more realistic. To form an effective confrontation strategy, the Graph Attention Multi-agent Soft Actor Critic Reinforcement Learning with Target Predicting Network (GA-MASAC-TP Net) is firstly proposed for Multi-UAV confrontation game. The merits lie in that the Multi-UAV trajectory prediction, considering interactions among targets, is incorporated innovatively into the Multi-agent reinforcement learning (MARL), enabling Multi-UAVs to make decisions more accurately based on situation prediction. Specifically, the Soft Actor Critic (SAC) algorithm is extended to the Multi-agent domain and embed with the graph attention neural network into the Actor, Critic network, so the UAV could aggregate the information of the spatial neighbor teammates based on the attention mechanism for better collaboration. The comparative experiment and ablation study demonstrate the effectiveness of the proposed algorithm and the state-of-art performance in the MCS.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140321690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Spiking neural networks (SNNs) are increasingly applied to deep architectures. Recent works are developed to apply spatio-temporal backpropagation to directly train deep SNNs. But the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing. In this paper, we first analyze the cause of the gradient vanishing problem and identify that the gradients mostly backpropagate along the synaptic currents. Based on that, we modify the synaptic current equation of leaky-integrate-fire neuron model and propose the improved LIF (IM-LIF) neuron model on the basis of the temporal-wise attention mechanism. We utilize the temporal-wise attention mechanism to selectively establish the connection between the current and historical response values, which can empirically enable the neuronal states to update resilient to the gradient vanishing problem. Furthermore, to capture the neuronal dynamics embedded in the output incorporating the IM-LIF model, we present a new temporal loss function to constrain the output of the network close to the target distribution. The proposed new temporal loss function could not only act as a regularizer to eliminate output outliers, but also assign the network loss credit to the voltage at a specific time point. Then we modify the ResNet and VGG architecture based on the IM-LIF model to build deep SNNs. We evaluate our work on image datasets and neuromorphic datasets. Experimental results and analysis show that our method can help build deep SNNs with competitive performance in both accuracy and latency, including 95.66% on CIFAR-10, 77.42% on CIFAR-100, 55.37% on Tiny-ImageNet, 97.33% on DVS-Gesture, and 80.50% on CIFAR-DVS with very few timesteps.
{"title":"IM-LIF: Improved Neuronal Dynamics With Attention Mechanism for Direct Training Deep Spiking Neural Network","authors":"Shuang Lian;Jiangrong Shen;Ziming Wang;Huajin Tang","doi":"10.1109/TETCI.2024.3359539","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3359539","url":null,"abstract":"Spiking neural networks (SNNs) are increasingly applied to deep architectures. Recent works are developed to apply spatio-temporal backpropagation to directly train deep SNNs. But the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing. In this paper, we first analyze the cause of the gradient vanishing problem and identify that the gradients mostly backpropagate along the synaptic currents. Based on that, we modify the synaptic current equation of leaky-integrate-fire neuron model and propose the improved LIF (IM-LIF) neuron model on the basis of the temporal-wise attention mechanism. We utilize the temporal-wise attention mechanism to selectively establish the connection between the current and historical response values, which can empirically enable the neuronal states to update resilient to the gradient vanishing problem. Furthermore, to capture the neuronal dynamics embedded in the output incorporating the IM-LIF model, we present a new temporal loss function to constrain the output of the network close to the target distribution. The proposed new temporal loss function could not only act as a regularizer to eliminate output outliers, but also assign the network loss credit to the voltage at a specific time point. Then we modify the ResNet and VGG architecture based on the IM-LIF model to build deep SNNs. We evaluate our work on image datasets and neuromorphic datasets. Experimental results and analysis show that our method can help build deep SNNs with competitive performance in both accuracy and latency, including 95.66% on CIFAR-10, 77.42% on CIFAR-100, 55.37% on Tiny-ImageNet, 97.33% on DVS-Gesture, and 80.50% on CIFAR-DVS with very few timesteps.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140321724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-13DOI: 10.1109/TETCI.2024.3360339
Chun-Na Li;Yi Li;Yuan-Hai Shao
Structured support vector machine (SSVM) is an effective method on coping with problems involving complex outputs such as multiple dependent output variables and structured output spaces. However, its training process is very time consuming for large-scale data with complex structure and many classes. In this paper, to improve the efficiency of SSVM, we propose a multiple structured support vector machine (MSSVM) for structured output classification via the idea of splitting large into small. By constructing novel classification loss for each class, MSSVM solves a series of smaller optimization problems rather than one large-size optimization problem in SSVM. Therefore, MSSVM greatly reduces the training speed of SSVM. In addition, the structured output label information and discriminative information are embedded in the introduced losses in a simple but effective way. Experiments on multiclass classification, ordinal regression and hierarchical classification datasets demonstrate the efficiency and effectiveness of the proposed MSSVM.
{"title":"Large-Scale Structured Output Classification via Multiple Structured Support Vector Machine by Splitting","authors":"Chun-Na Li;Yi Li;Yuan-Hai Shao","doi":"10.1109/TETCI.2024.3360339","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3360339","url":null,"abstract":"Structured support vector machine (SSVM) is an effective method on coping with problems involving complex outputs such as multiple dependent output variables and structured output spaces. However, its training process is very time consuming for large-scale data with complex structure and many classes. In this paper, to improve the efficiency of SSVM, we propose a multiple structured support vector machine (MSSVM) for structured output classification via the idea of splitting large into small. By constructing novel classification loss for each class, MSSVM solves a series of smaller optimization problems rather than one large-size optimization problem in SSVM. Therefore, MSSVM greatly reduces the training speed of SSVM. In addition, the structured output label information and discriminative information are embedded in the introduced losses in a simple but effective way. Experiments on multiclass classification, ordinal regression and hierarchical classification datasets demonstrate the efficiency and effectiveness of the proposed MSSVM.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140321735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：