Pub Date : 2024-10-07DOI: 10.1109/TAI.2024.3474654
Yang Yang;Chao Wang;Lei Gong;Min Wu;Zhenghua Chen;Xiang Li;Xianglan Chen;Xuehai Zhou
Ensemble object detectors have demonstrated remarkable effectiveness in enhancing prediction accuracy and uncertainty quantification. However, their widespread adoption is hindered by significant computational and storage demands, limiting their feasibility in resource-constrained settings. To overcome this, researchers have focused on distilling the knowledge from ensemble object detectors into a single model. In this article, we introduce probabilization based ensemble distillation (ProbED), an innovative ensemble distillation framework that consolidates knowledge from multiple object detectors into a single, resource-efficient model. Unlike traditional ensemble distillation methods that average the outputs of subteachers, ProbED captures comprehensive outcome distributions from all subteachers, providing a more nuanced approach to knowledge transfer. ProbED employs knowledge probabilization to achieve a sophisticated and refined aggregation of teacher knowledge, including feature knowledge, semantic knowledge, and localization knowledge, resulting in dual improvements in prediction accuracy and uncertainty quantification for the student model. In particular, ProED's novel knowledge probabilization-based approach to aggregating teacher knowledge is inspired by our empirical observations, which demonstrate that knowledge probabilization excels in effectively representing uncertainty, improving prediction, and facilitating robust knowledge transfer. Furthermore, we introduce a random smoothing perturbation technique to modify inputs within ProbED, further enhancing the distillation process. Extensive experiments highlight ProbED's ability to significantly enhance the prediction accuracy and uncertainty quantification of various object detectors, demonstrating its superior performance compared to other state-of-the-art techniques.
{"title":"Knowledge Probabilization in Ensemble Distillation: Improving Accuracy and Uncertainty Quantification for Object Detectors","authors":"Yang Yang;Chao Wang;Lei Gong;Min Wu;Zhenghua Chen;Xiang Li;Xianglan Chen;Xuehai Zhou","doi":"10.1109/TAI.2024.3474654","DOIUrl":"https://doi.org/10.1109/TAI.2024.3474654","url":null,"abstract":"Ensemble object detectors have demonstrated remarkable effectiveness in enhancing prediction accuracy and uncertainty quantification. However, their widespread adoption is hindered by significant computational and storage demands, limiting their feasibility in resource-constrained settings. To overcome this, researchers have focused on distilling the knowledge from ensemble object detectors into a single model. In this article, we introduce probabilization based ensemble distillation (ProbED), an innovative ensemble distillation framework that consolidates knowledge from multiple object detectors into a single, resource-efficient model. Unlike traditional ensemble distillation methods that average the outputs of subteachers, ProbED captures comprehensive outcome distributions from all subteachers, providing a more nuanced approach to knowledge transfer. ProbED employs knowledge probabilization to achieve a sophisticated and refined aggregation of teacher knowledge, including feature knowledge, semantic knowledge, and localization knowledge, resulting in dual improvements in prediction accuracy and uncertainty quantification for the student model. In particular, ProED's novel knowledge probabilization-based approach to aggregating teacher knowledge is inspired by our empirical observations, which demonstrate that knowledge probabilization excels in effectively representing uncertainty, improving prediction, and facilitating robust knowledge transfer. Furthermore, we introduce a random smoothing perturbation technique to modify inputs within ProbED, further enhancing the distillation process. Extensive experiments highlight ProbED's ability to significantly enhance the prediction accuracy and uncertainty quantification of various object detectors, demonstrating its superior performance compared to other state-of-the-art techniques.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"221-233"},"PeriodicalIF":0.0,"publicationDate":"2024-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Research into fake news detection has a long history, although it gained significant attention following the 2016 U.S. election. During this time, the widespread use of social media and the resulting increase in interpersonal communication led to the extensive spread of ambiguous and potentially misleading news. Traditional approaches, relying solely on pre-large language model (LLM) techniques and addressing the issue as a simple classification problem, have shown to be insufficient for improving detection accuracy. In this context, LLMs have become crucial, as their advanced architectures overcome the limitations of pre-LLM methods, which often fail to capture the subtleties of fake news. This literature review aims to shed light on the field of fake news detection by providing a brief historical overview, defining fake news, reviewing detection methods used before the advent of LLMs, and discussing the strengths and weaknesses of these models in an increasingly complex landscape. Furthermore, it will emphasize the importance of using multimodal datasets in the effort to detect fake news.
{"title":"Under the Influence: A Survey of Large Language Models in Fake News Detection","authors":"Soveatin Kuntur;Anna Wróblewska;Marcin Paprzycki;Maria Ganzha","doi":"10.1109/TAI.2024.3471735","DOIUrl":"https://doi.org/10.1109/TAI.2024.3471735","url":null,"abstract":"Research into fake news detection has a long history, although it gained significant attention following the 2016 U.S. election. During this time, the widespread use of social media and the resulting increase in interpersonal communication led to the extensive spread of ambiguous and potentially misleading news. Traditional approaches, relying solely on pre-large language model (LLM) techniques and addressing the issue as a simple classification problem, have shown to be insufficient for improving detection accuracy. In this context, LLMs have become crucial, as their advanced architectures overcome the limitations of pre-LLM methods, which often fail to capture the subtleties of fake news. This literature review aims to shed light on the field of fake news detection by providing a brief historical overview, defining fake news, reviewing detection methods used before the advent of LLMs, and discussing the strengths and weaknesses of these models in an increasingly complex landscape. Furthermore, it will emphasize the importance of using multimodal datasets in the effort to detect fake news.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 2","pages":"458-476"},"PeriodicalIF":0.0,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143535459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the growing popularity of the Internet, Web applications have become increasingly essential in our daily lives. Web application programming interfaces (Web APIs) play a crucial role in facilitating interaction between applications. However, most Web service platforms are suffering from the imbalance of Web services now, many services of good quality but low popularity are difficult to be invoked even once and do not create direct connections with the users. Some graph-based Web service recommendation methods also often present a long-tailed distribution of recommended Web services due to limited Mashup–API invocation relationships. To relieve this problem and promote service recommendation, in this article, we propose a community structure and popularity-based approach by constructing an evolving cooperation network for Web APIs. We leverage the Louvain algorithm in community detection to assign community structure to each Web API and consider both the popularity and community structure in constructing the network. By optimizing the Barabάsi–Albert (BA) evolving network model, we demonstrate that our approach outperforms the BA, Bianconi–Barabάsi (BB), and popularity-similarity optimization (PSO) models in Web service clustering. Based on our proposed evolutionary network model for the evolutionary extension of API cooperation network and used for downstream Web service recommendation tasks, the experimental results also show that our recommended approach outperforms some other baseline models for Web service recommendation.
随着Internet的日益普及，Web应用程序在我们的日常生活中变得越来越重要。Web应用程序编程接口（Web api）在促进应用程序之间的交互方面起着至关重要的作用。然而，目前大多数Web服务平台都存在着Web服务不均衡的问题，许多质量好的但知名度不高的服务甚至很难被调用一次，也无法与用户建立直接连接。由于Mashup-API调用关系有限，一些基于图的Web服务推荐方法也经常呈现推荐的Web服务的长尾分布。为了缓解这一问题并促进服务推荐，本文通过构建Web api的演进合作网络，提出了一种基于社区结构和流行度的方法。我们利用社区检测中的Louvain算法为每个Web API分配社区结构，并在构建网络时考虑流行度和社区结构。通过优化barab si - albert （BA）进化网络模型，我们证明了我们的方法在Web服务聚类中优于BA、bianconi - barab si （BB）和流行度相似度优化（PSO）模型。基于我们提出的API协作网络进化扩展的进化网络模型，并将其用于下游Web服务推荐任务，实验结果还表明，我们的推荐方法优于其他一些Web服务推荐基线模型。
{"title":"Evolution of Web API Cooperation Network via Exploring Community Structure and Popularity","authors":"Guosheng Kang;Yang Wang;Jianxun Liu;Buqing Cao;Yong Xiao;Yu Xu","doi":"10.1109/TAI.2024.3472614","DOIUrl":"https://doi.org/10.1109/TAI.2024.3472614","url":null,"abstract":"With the growing popularity of the Internet, Web applications have become increasingly essential in our daily lives. Web application programming interfaces (Web APIs) play a crucial role in facilitating interaction between applications. However, most Web service platforms are suffering from the imbalance of Web services now, many services of good quality but low popularity are difficult to be invoked even once and do not create direct connections with the users. Some graph-based Web service recommendation methods also often present a long-tailed distribution of recommended Web services due to limited Mashup–API invocation relationships. To relieve this problem and promote service recommendation, in this article, we propose a community structure and popularity-based approach by constructing an evolving cooperation network for Web APIs. We leverage the Louvain algorithm in community detection to assign community structure to each Web API and consider both the popularity and community structure in constructing the network. By optimizing the Barabάsi–Albert (BA) evolving network model, we demonstrate that our approach outperforms the BA, Bianconi–Barabάsi (BB), and popularity-similarity optimization (PSO) models in Web service clustering. Based on our proposed evolutionary network model for the evolutionary extension of API cooperation network and used for downstream Web service recommendation tasks, the experimental results also show that our recommended approach outperforms some other baseline models for Web service recommendation.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6659-6671"},"PeriodicalIF":0.0,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-02DOI: 10.1109/TAI.2024.3471729
Zhiguo Yan;Tingkun Sun;Guolin Hu
This article investigates the infinite horizon optimal control problem for stochastic Markovian jump systems with Wiener and Poisson noises. First, a new policy iteration algorithm is designed by using integral reinforcement learning approach and subsystems transformation technique, which obtains the optimal solution without solving stochastic coupled algebraic Riccati equation (SCARE) directly. Second, through the transformation and substitution of the SCARE and feedback gain matrix, a policy iteration algorithm is devised to determine the optimal control strategy. This algorithm leverages only state trajectory information to obtain the optimal solution, and is updated in an unfixed form. Additionally, the algorithm remains unaffected by variations in Poisson jump intensity. Finally, an numerical example is given to verify the effectiveness and convergence of the proposed algorithms.
{"title":"Optimal Control of Stochastic Markovian Jump Systems With Wiener and Poisson Noises: Two Reinforcement Learning Approaches","authors":"Zhiguo Yan;Tingkun Sun;Guolin Hu","doi":"10.1109/TAI.2024.3471729","DOIUrl":"https://doi.org/10.1109/TAI.2024.3471729","url":null,"abstract":"This article investigates the infinite horizon optimal control problem for stochastic Markovian jump systems with Wiener and Poisson noises. First, a new policy iteration algorithm is designed by using integral reinforcement learning approach and subsystems transformation technique, which obtains the optimal solution without solving stochastic coupled algebraic Riccati equation (SCARE) directly. Second, through the transformation and substitution of the SCARE and feedback gain matrix, a policy iteration algorithm is devised to determine the optimal control strategy. This algorithm leverages only state trajectory information to obtain the optimal solution, and is updated in an unfixed form. Additionally, the algorithm remains unaffected by variations in Poisson jump intensity. Finally, an numerical example is given to verify the effectiveness and convergence of the proposed algorithms.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6591-6600"},"PeriodicalIF":0.0,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825813","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TAI.2024.3465441
Manaar Alam;Hithem Lamri;Michail Maniatakos
Federated learning (FL) enables collaborative learning across multiple participants without exposing sensitive personal data. However, the distributed nature of FL and unvetted participants’ data makes it vulnerable to backdoor attacks . In these attacks, adversaries selectively inject malicious functionality into the centralized model during training, leading to intentional misclassifications for specific adversary-chosen inputs. While previous research has demonstrated successful injections of persistent backdoors in FL, the persistence also poses a challenge, as their existence in the centralized model can prompt the central aggregation server to take preventive measures for penalizing the adversaries. Therefore, this article proposes a method that enables adversaries to effectively remove backdoors from the centralized model upon achieving their objectives or upon suspicion of possible detection. The proposed approach extends the concept of machine unlearning and presents strategies to preserve the performance of the centralized model and simultaneously prevent over-unlearning of information unrelated to backdoor patterns, making adversaries stealthy while removing backdoors. To the best of our knowledge, this is the first work exploring machine unlearning in FL to remove backdoors to the benefit of adversaries. Exhaustive evaluation considering various image classification scenarios demonstrates the efficacy of the proposed method for efficient backdoor removal from the centralized model, injected by state-of-the-art attacks across multiple configurations.
{"title":"Get Rid of Your Trail: Remotely Erasing Backdoors in Federated Learning","authors":"Manaar Alam;Hithem Lamri;Michail Maniatakos","doi":"10.1109/TAI.2024.3465441","DOIUrl":"https://doi.org/10.1109/TAI.2024.3465441","url":null,"abstract":"Federated learning (FL) enables collaborative learning across multiple participants without exposing sensitive personal data. However, the distributed nature of FL and unvetted participants’ data makes it vulnerable to \u0000<italic>backdoor attacks</i>\u0000. In these attacks, adversaries selectively inject malicious functionality into the centralized model during training, leading to intentional misclassifications for specific adversary-chosen inputs. While previous research has demonstrated successful injections of persistent backdoors in FL, the persistence also poses a challenge, as their existence in the centralized model can prompt the central aggregation server to take preventive measures for penalizing the adversaries. Therefore, this article proposes a method \u0000<italic>that enables adversaries to effectively remove backdoors from the centralized model</i>\u0000 upon achieving their objectives or upon suspicion of possible detection. The proposed approach extends the concept of \u0000<italic>machine unlearning</i>\u0000 and presents strategies to preserve the performance of the centralized model and simultaneously prevent over-unlearning of information unrelated to backdoor patterns, making adversaries stealthy while removing backdoors. To the best of our knowledge, this is the first work exploring machine unlearning in FL to remove backdoors to the benefit of adversaries. Exhaustive evaluation considering various image classification scenarios demonstrates the efficacy of the proposed method for efficient backdoor removal from the centralized model, injected by state-of-the-art attacks across multiple configurations.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6683-6698"},"PeriodicalIF":0.0,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TAI.2024.3465433
Yu-Ming Zhang;Jun-Wei Hsieh;Chun-Chieh Lee;Kuo-Chin Fan
Manually designed convolutional neural networks (CNNs) architectures such as visual geometry group network (VGG), ResNet, DenseNet, and MobileNet have achieved high performance across various tasks, but design them is time-consuming and costly. Neural architecture search (NAS) automates the discovery of effective CNN architectures, reducing the need for experts. However, evaluating candidate architectures requires significant graphics processing unit (GPU) resources, leading to the use of predictor-based NAS, such as graph convolutional networks (GCN), which is the popular option to construct predictors. However, we discover that, even though the ability of GCN mimics the propagation of features of real architectures, the binary nature of the adjacency matrix limits its effectiveness. To address this, we propose redirection of adjacent trails (RATs), which adaptively learns trail weights within the adjacency matrix. Our RATs-GCN outperform other predictors by dynamically adjusting trail weights after each graph convolution layer. Additionally, the proposed divide search sampling (DSS) strategy, based on the observation of cell-based NAS that architectures with similar floating point operations (FLOPs) perform similarly, enhances search efficiency. Our RATs-NAS, which combine RATs-GCN and DSS, shows significant improvements over other predictor-based NAS methods on NASBench-101, NASBench-201, and NASBench-301.
{"title":"RATs-NAS: Redirection of Adjacent Trails on Graph Convolutional Networks for Predictor-Based Neural Architecture Search","authors":"Yu-Ming Zhang;Jun-Wei Hsieh;Chun-Chieh Lee;Kuo-Chin Fan","doi":"10.1109/TAI.2024.3465433","DOIUrl":"https://doi.org/10.1109/TAI.2024.3465433","url":null,"abstract":"Manually designed convolutional neural networks (CNNs) architectures such as visual geometry group network (VGG), ResNet, DenseNet, and MobileNet have achieved high performance across various tasks, but design them is time-consuming and costly. Neural architecture search (NAS) automates the discovery of effective CNN architectures, reducing the need for experts. However, evaluating candidate architectures requires significant graphics processing unit (GPU) resources, leading to the use of predictor-based NAS, such as graph convolutional networks (GCN), which is the popular option to construct predictors. However, we discover that, even though the ability of GCN mimics the propagation of features of real architectures, the binary nature of the adjacency matrix limits its effectiveness. To address this, we propose redirection of adjacent trails (RATs), which adaptively learns trail weights within the adjacency matrix. Our RATs-GCN outperform other predictors by dynamically adjusting trail weights after each graph convolution layer. Additionally, the proposed divide search sampling (DSS) strategy, based on the observation of cell-based NAS that architectures with similar floating point operations (FLOPs) perform similarly, enhances search efficiency. Our RATs-NAS, which combine RATs-GCN and DSS, shows significant improvements over other predictor-based NAS methods on NASBench-101, NASBench-201, and NASBench-301.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6672-6682"},"PeriodicalIF":0.0,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Smart power grids are vulnerable to security threats due to their cyber-physical nature. Existing data-driven detectors aim to address simple traditional false data injection attacks (FDIAs). However, adversarial false data injection evasion attacks (FDIEAs) present a more serious threat as adversaries, with different levels of knowledge about the system, inject adversarial samples to circumvent the grid's attack detection system. The robustness of state-of-the-art graph-based detectors has not been investigated against sophisticated FDIEAs. Hence, this article answers three research questions. 1) What is the impact of utilizing spatio-temporal features to craft adversarial samples and how to select attack nodes? 2) How can adversaries generate surrogate spatio-temporal data when they lack knowledge about the system topology? 3) What are the required model characteristics for a robust detection against adversarial FDIEAs? To answer the questions, we examine the robustness of several detectors against five attack cases and conclude the following: 1) Attack generation with full knowledge using spatio-temporal features leads to 5%–26% and 2%–5% higher degradation in detection rate (DR) compared to traditional FDIAs and using temporal features, respectively, whereas centrality analysis-based attack node selection leads to 3%–11% higher degradation in DR compared to a random selection; 2) Stochastic geometry-based graph generation to create surrogate adversarial topologies and samples leads to 3%–13% higher degradation in DR compared to traditional FDIAs; and 3) Adopting an unsupervised spatio-temporal graph autoencoder (STGAE)-based detector enhances the DR by 5 $-$ 53% compared to benchmark detectors against FDIEAs.
{"title":"Spatio-temporal Graph-Based Generation and Detection of Adversarial False Data Injection Evasion Attacks in Smart Grids","authors":"Abdulrahman Takiddin;Muhammad Ismail;Rachad Atat;Erchin Serpedin","doi":"10.1109/TAI.2024.3464511","DOIUrl":"https://doi.org/10.1109/TAI.2024.3464511","url":null,"abstract":"Smart power grids are vulnerable to security threats due to their cyber-physical nature. Existing data-driven detectors aim to address simple traditional false data injection attacks (FDIAs). However, adversarial false data injection evasion attacks (FDIEAs) present a more serious threat as adversaries, with different levels of knowledge about the system, inject adversarial samples to circumvent the grid's attack detection system. The robustness of state-of-the-art graph-based detectors has not been investigated against sophisticated FDIEAs. Hence, this article answers three research questions. 1) What is the impact of utilizing spatio-temporal features to craft adversarial samples and how to select attack nodes? 2) How can adversaries generate surrogate spatio-temporal data when they lack knowledge about the system topology? 3) What are the required model characteristics for a robust detection against adversarial FDIEAs? To answer the questions, we examine the robustness of several detectors against five attack cases and conclude the following: 1) Attack generation with full knowledge using spatio-temporal features leads to 5%–26% and 2%–5% higher degradation in detection rate (DR) compared to traditional FDIAs and using temporal features, respectively, whereas centrality analysis-based attack node selection leads to 3%–11% higher degradation in DR compared to a random selection; 2) Stochastic geometry-based graph generation to create surrogate adversarial topologies and samples leads to 3%–13% higher degradation in DR compared to traditional FDIAs; and 3) Adopting an unsupervised spatio-temporal graph autoencoder (STGAE)-based detector enhances the DR by 5\u0000<inline-formula><tex-math>$-$</tex-math></inline-formula>\u000053% compared to benchmark detectors against FDIEAs.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6601-6616"},"PeriodicalIF":0.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TAI.2024.3464510
Yuhang Zhang;Chao Yan;Jiaping Xiao;Mir Feroskhan
Obstacle avoidance under constrained visual perception presents a significant challenge, requiring rapid detection and decision-making within partially observable environments, particularly for unmanned aerial vehicles (UAVs) maneuvering agilely in 3-D space. Compared with traditional methods, obstacle avoidance algorithms based on deep reinforcement learning (DRL) offer a better comprehension of the uncertain operational environment in an end-to-end manner, reducing computational complexity, and enhancing flexibility and scalability. However, the inherent trial-and-error learning mechanism of DRL necessitates numerous iterations for policy convergence, leading to sample inefficiency issues. Meanwhile, existing sample-efficient obstacle avoidance approaches that leverage imitation learning often heavily rely on offline expert demonstrations, which are not always feasible in hazardous environments. To address these challenges, we propose a novel obstacle avoidance approach based on nonexpert policy enhanced DRL (NPE-DRL). This approach integrates a fundamental DRL framework with prior knowledge derived from a nonexpert policy-guided imitation learning. During the training phase, the agent starts by online imitating the actions generated by the nonexpert policy during interactions and progressively shifts toward autonomously exploring the environment to generate the optimal policy. Both simulation and physical experiments validate that our approach improves sample efficiency and achieves a better exploration–exploitation balance in both virtual and real-world flights. Additionally, our NPE-DRL-based obstacle avoidance approach shows better adaptability in complex environments characterized by larger scales and denser obstacle configurations, demonstrating a significant improvement in UAVs’ obstacle avoidance capability. Code available at https://github.com/zzzzzyh111/NonExpert-Guided-Visual-UAV-Navigation-Gazebo.
{"title":"NPE-DRL: Enhancing Perception Constrained Obstacle Avoidance With Nonexpert Policy Guided Reinforcement Learning","authors":"Yuhang Zhang;Chao Yan;Jiaping Xiao;Mir Feroskhan","doi":"10.1109/TAI.2024.3464510","DOIUrl":"https://doi.org/10.1109/TAI.2024.3464510","url":null,"abstract":"Obstacle avoidance under constrained visual perception presents a significant challenge, requiring rapid detection and decision-making within partially observable environments, particularly for unmanned aerial vehicles (UAVs) maneuvering agilely in 3-D space. Compared with traditional methods, obstacle avoidance algorithms based on deep reinforcement learning (DRL) offer a better comprehension of the uncertain operational environment in an end-to-end manner, reducing computational complexity, and enhancing flexibility and scalability. However, the inherent trial-and-error learning mechanism of DRL necessitates numerous iterations for policy convergence, leading to sample inefficiency issues. Meanwhile, existing sample-efficient obstacle avoidance approaches that leverage imitation learning often heavily rely on offline expert demonstrations, which are not always feasible in hazardous environments. To address these challenges, we propose a novel obstacle avoidance approach based on nonexpert policy enhanced DRL (NPE-DRL). This approach integrates a fundamental DRL framework with prior knowledge derived from a nonexpert policy-guided imitation learning. During the training phase, the agent starts by online imitating the actions generated by the nonexpert policy during interactions and progressively shifts toward autonomously exploring the environment to generate the optimal policy. Both simulation and physical experiments validate that our approach improves sample efficiency and achieves a better exploration–exploitation balance in both virtual and real-world flights. Additionally, our NPE-DRL-based obstacle avoidance approach shows better adaptability in complex environments characterized by larger scales and denser obstacle configurations, demonstrating a significant improvement in UAVs’ obstacle avoidance capability. Code available at <uri>https://github.com/zzzzzyh111/NonExpert-Guided-Visual-UAV-Navigation-Gazebo</uri>.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 1","pages":"184-198"},"PeriodicalIF":0.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TAI.2024.3461630
Tianyu Zhu;Xinli Shi;Xiangping Xu;Jinde Cao
The utilization of neural network models for solving combinatorial optimization problems (COPs) has gained significant attention in recent years and has demonstrated encouraging outcomes in addressing analogous problems such as the traveling salesman problem (TSP). The multiple TSP (MTSP) has sparked the interest of researchers as a special kind of COPs. The colored TSP (CTSP) is a variation of the MTSP, which utilizes colors to distinguish the accessibility of cities to salesmen. This article proposes a gated entropy-insensitive attention model (GEIAM) to solve CTSP. In specific, the original problem is first modeled as a sequence and preprocessed by the problem feature extraction network of the model, and then solved by the autoregressive solution constructor subsequently. The policy (parameters of the neural network model) is trained via reinforcement learning (RL). The proposed approach is compared with several commercial solvers as well as heuristics and demonstrates superior solving speed with comparable solution quality.
{"title":"Reinforcement Learning for Solving Colored Traveling Salesman Problems: An Entropy-Insensitive Attention Approach","authors":"Tianyu Zhu;Xinli Shi;Xiangping Xu;Jinde Cao","doi":"10.1109/TAI.2024.3461630","DOIUrl":"https://doi.org/10.1109/TAI.2024.3461630","url":null,"abstract":"The utilization of neural network models for solving combinatorial optimization problems (COPs) has gained significant attention in recent years and has demonstrated encouraging outcomes in addressing analogous problems such as the traveling salesman problem (TSP). The multiple TSP (MTSP) has sparked the interest of researchers as a special kind of COPs. The colored TSP (CTSP) is a variation of the MTSP, which utilizes colors to distinguish the accessibility of cities to salesmen. This article proposes a gated entropy-insensitive attention model (GEIAM) to solve CTSP. In specific, the original problem is first modeled as a sequence and preprocessed by the problem feature extraction network of the model, and then solved by the autoregressive solution constructor subsequently. The policy (parameters of the neural network model) is trained via reinforcement learning (RL). The proposed approach is compared with several commercial solvers as well as heuristics and demonstrates superior solving speed with comparable solution quality.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6699-6708"},"PeriodicalIF":0.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-18DOI: 10.1109/TAI.2024.3433614
Prachi Pratyusha Sahoo;Kyriakos G. Vamvoudakis
In this article, we provide a model-free reinforcement learning (RL) framework that relies on internal reinforcement signals, called self-model-free RL, for learning agents that experience loss of the reinforcement signals in the form of packet drops and/or jamming attacks by malicious agents. The framework embeds a correcting mechanism in the form of a goal network to compensate for information loss and produce optimal and stabilizing policies. It also provides a trade-off scheme that reconstructs the reward using a goal network whenever the reinforcement signals are lost but utilizes true reinforcement signals when they are available. The stability of the equilibrium point is guaranteed despite fractional information loss in the reinforcement signals. Finally, simulation results validate the efficacy of the proposed work.
{"title":"Self-Model-Free Learning Versus Learning With External Rewards in Information Constrained Environments","authors":"Prachi Pratyusha Sahoo;Kyriakos G. Vamvoudakis","doi":"10.1109/TAI.2024.3433614","DOIUrl":"https://doi.org/10.1109/TAI.2024.3433614","url":null,"abstract":"In this article, we provide a model-free reinforcement learning (RL) framework that relies on internal reinforcement signals, called self-model-free RL, for learning agents that experience loss of the reinforcement signals in the form of packet drops and/or jamming attacks by malicious agents. The framework embeds a correcting mechanism in the form of a goal network to compensate for information loss and produce optimal and stabilizing policies. It also provides a trade-off scheme that reconstructs the reward using a goal network whenever the reinforcement signals are lost but utilizes true reinforcement signals when they are available. The stability of the equilibrium point is guaranteed despite fractional information loss in the reinforcement signals. Finally, simulation results validate the efficacy of the proposed work.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"5 12","pages":"6566-6579"},"PeriodicalIF":0.0,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：