Pub Date : 2024-03-16DOI: 10.1109/OJVT.2024.3402129
Zhipeng Wang;Soon Xin Ng;Mohammed EI-Hajjar
In recent years, unmanned aerial vehicles (UAVs) have been considered for many applications, such as disaster prevention and control, logistics and transportation, and wireless communication. Most UAVs need to be manually controlled using remote control, which can be challenging in many environments. Therefore, autonomous UAVs have attracted significant research interest, where most of the existing autonomous navigation algorithms suffer from long computation time and unsatisfactory performance. Hence, we propose a Deep Reinforcement Learning (DRL) UAV path planning algorithm based on cumulative reward and region segmentation. Our proposed region segmentation aims to reduce the probability of DRL agents falling into local optimal trap, while our proposed cumulative reward model takes into account the distance from the node to the destination and the density of obstacles near the node, which solves the problem of sparse training data faced by the DRL algorithms in the path planning task. The proposed region segmentation algorithm and cumulative reward model have been tested in different DRL techniques, where we show that the cumulative reward model can improve the training efficiency of deep neural networks by 30.8% and the region segmentation algorithm enables deep Q-network agent to avoid 99% of local optimal traps and assists deep deterministic policy gradient agent to avoid 92% of local optimal traps.
{"title":"Deep Reinforcement Learning Assisted UAV Path Planning Relying on Cumulative Reward Mode and Region Segmentation","authors":"Zhipeng Wang;Soon Xin Ng;Mohammed EI-Hajjar","doi":"10.1109/OJVT.2024.3402129","DOIUrl":"https://doi.org/10.1109/OJVT.2024.3402129","url":null,"abstract":"In recent years, unmanned aerial vehicles (UAVs) have been considered for many applications, such as disaster prevention and control, logistics and transportation, and wireless communication. Most UAVs need to be manually controlled using remote control, which can be challenging in many environments. Therefore, autonomous UAVs have attracted significant research interest, where most of the existing autonomous navigation algorithms suffer from long computation time and unsatisfactory performance. Hence, we propose a Deep Reinforcement Learning (DRL) UAV path planning algorithm based on cumulative reward and region segmentation. Our proposed region segmentation aims to reduce the probability of DRL agents falling into local optimal trap, while our proposed cumulative reward model takes into account the distance from the node to the destination and the density of obstacles near the node, which solves the problem of sparse training data faced by the DRL algorithms in the path planning task. The proposed region segmentation algorithm and cumulative reward model have been tested in different DRL techniques, where we show that the cumulative reward model can improve the training efficiency of deep neural networks by 30.8% and the region segmentation algorithm enables deep Q-network agent to avoid 99% of local optimal traps and assists deep deterministic policy gradient agent to avoid 92% of local optimal traps.","PeriodicalId":34270,"journal":{"name":"IEEE Open Journal of Vehicular Technology","volume":null,"pages":null},"PeriodicalIF":6.4,"publicationDate":"2024-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10531630","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141304035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper provides a comprehensive overview of the evolution of Machine Learning (ML), from traditional to advanced, in its application and integration into unmanned aerial vehicle (UAV) communication frameworks and practical applications. The manuscript starts with an overview of the existing research on UAV communication and introduces the most traditional ML techniques. It then discusses UAVs as versatile actors in mobile networks, assuming different roles from airborne user equipment (UE) to base stations (BS). UAV have demonstrated considerable potential in addressing the evolving challenges of next-generation mobile networks, such as enhancing coverage and facilitating temporary hotspots but pose new hurdles including optimal positioning, trajectory optimization, and energy efficiency. We therefore conduct a comprehensive review of advanced ML strategies, ranging from federated learning, transfer and meta-learning to explainable AI, to address those challenges. Finally, the use of state-of-the-art ML algorithms in these capabilities is explored and their potential extension to cloud and/or edge computing based network architectures is highlighted.
本文全面概述了机器学习(ML)从传统到先进的演变过程,及其在无人机(UAV)通信框架和实际应用中的应用和集成。文稿首先概述了无人机通信方面的现有研究,并介绍了最传统的 ML 技术。然后讨论了无人机作为移动网络中的多面手,承担着从机载用户设备(UE)到基站(BS)的不同角色。无人机在应对下一代移动网络不断发展的挑战(如增强覆盖范围和促进临时热点)方面表现出了相当大的潜力,但也带来了新的障碍,包括优化定位、轨迹优化和能效。因此,我们全面回顾了先进的人工智能策略,从联合学习、迁移学习和元学习到可解释人工智能,以应对这些挑战。最后,我们探讨了最先进的人工智能算法在这些能力中的应用,并强调了这些算法扩展到基于云计算和/或边缘计算的网络架构的潜力。
{"title":"Advancing UAV Communications: A Comprehensive Survey of Cutting-Edge Machine Learning Techniques","authors":"Chenrui Sun;Gianluca Fontanesi;Berk Canberk;Amirhossein Mohajerzadeh;Symeon Chatzinotas;David Grace;Hamed Ahmadi","doi":"10.1109/OJVT.2024.3401024","DOIUrl":"https://doi.org/10.1109/OJVT.2024.3401024","url":null,"abstract":"This paper provides a comprehensive overview of the evolution of Machine Learning (ML), from traditional to advanced, in its application and integration into unmanned aerial vehicle (UAV) communication frameworks and practical applications. The manuscript starts with an overview of the existing research on UAV communication and introduces the most traditional ML techniques. It then discusses UAVs as versatile actors in mobile networks, assuming different roles from airborne user equipment (UE) to base stations (BS). UAV have demonstrated considerable potential in addressing the evolving challenges of next-generation mobile networks, such as enhancing coverage and facilitating temporary hotspots but pose new hurdles including optimal positioning, trajectory optimization, and energy efficiency. We therefore conduct a comprehensive review of advanced ML strategies, ranging from federated learning, transfer and meta-learning to explainable AI, to address those challenges. Finally, the use of state-of-the-art ML algorithms in these capabilities is explored and their potential extension to cloud and/or edge computing based network architectures is highlighted.","PeriodicalId":34270,"journal":{"name":"IEEE Open Journal of Vehicular Technology","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10531095","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141624107","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1109/OJVT.2024.3400901
Pedro H. D. Almeida;Hugerles S. Silva;Ugo S. Dias;Rausley A. A. de Souza;Iguatemi E. Fonseca;Yonghui Li
In this article, exact expressions are presented for the probability density function, cumulative distribution function, moment generating function, and higher-order moments of the instantaneous signal-to-noise ratio, considering the product and the sum of the products of $N$