Unmanned aerial vehicles (UAVs) play a crucial role in various domains such as military, civil, industrial, and so on. However, the coordination and task allocation of multiple UAVs in engineering practices face numerous challenges. As the scale of battlefields expands, and the diversity of UAV missions and constraints increases, the existing task allocation methods suffer from issues such as a mismatch between theoretical models and real-world applications, low task execution efficiency, and poor responsiveness in dynamic environments. To address these challenges, this paper proposes an improved genetic algorithm (GA)-based approach for multi-UAV cooperative task allocation. By collecting battlefield information, decomposing tasks, and considering UAV resource types, an optimization model for multi-UAV cooperative task allocation is constructed. The proposed method, using an improved GA, generates a set of Pareto-optimal task solutions for decision-makers. Case studies demonstrate that this approach effectively enhances task execution efficiency and reduces the total flight distance cost of UAVs.
{"title":"Cooperative task allocation method for multi-unmanned aerial vehicles based on the modified genetic algorithm","authors":"Yifang Tan, Chao Zhou, Feng Qian","doi":"10.1049/itr2.12495","DOIUrl":"10.1049/itr2.12495","url":null,"abstract":"<p>Unmanned aerial vehicles (UAVs) play a crucial role in various domains such as military, civil, industrial, and so on. However, the coordination and task allocation of multiple UAVs in engineering practices face numerous challenges. As the scale of battlefields expands, and the diversity of UAV missions and constraints increases, the existing task allocation methods suffer from issues such as a mismatch between theoretical models and real-world applications, low task execution efficiency, and poor responsiveness in dynamic environments. To address these challenges, this paper proposes an improved genetic algorithm (GA)-based approach for multi-UAV cooperative task allocation. By collecting battlefield information, decomposing tasks, and considering UAV resource types, an optimization model for multi-UAV cooperative task allocation is constructed. The proposed method, using an improved GA, generates a set of Pareto-optimal task solutions for decision-makers. Case studies demonstrate that this approach effectively enhances task execution efficiency and reduces the total flight distance cost of UAVs.</p>","PeriodicalId":50381,"journal":{"name":"IET Intelligent Transport Systems","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/itr2.12495","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139764472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Most current RM approaches are developed for fixed bottlenecks. However, the number and locations of bottlenecks are usually uncertain and even time-varying due to some unexpected phenomena, such as severe accidents and temporal lane closures. Thus, the RM approach should be able to enhance traffic flow stability by effectively handling the time-delay effect and fluctuations in traffic flow rate caused by uncertain bottlenecks. This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on-ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on-ramp bottleneck, fixed downstream bottleneck, random-location bottleneck, and multiple bottlenecks. With curriculum learning, the entire training process was reduced by 45.1% to 64.5%, while maintaining a similar maximum reward level compared to DRL-based RM control with full learning from scratch. Specifically, the results also demonstrated that the proposed DRLCL-based RM outperformed the feedback-based RM due to its stronger predictive ability, faster response, and higher action precision.
{"title":"Enhancing reinforcement learning-based ramp metering performance at freeway uncertain bottlenecks using curriculum learning","authors":"Si Zheng, Zhibin Li, Meng Li, Zemian Ke","doi":"10.1049/itr2.12494","DOIUrl":"10.1049/itr2.12494","url":null,"abstract":"<p>Most current RM approaches are developed for fixed bottlenecks. However, the number and locations of bottlenecks are usually uncertain and even time-varying due to some unexpected phenomena, such as severe accidents and temporal lane closures. Thus, the RM approach should be able to enhance traffic flow stability by effectively handling the time-delay effect and fluctuations in traffic flow rate caused by uncertain bottlenecks. This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on-ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on-ramp bottleneck, fixed downstream bottleneck, random-location bottleneck, and multiple bottlenecks. With curriculum learning, the entire training process was reduced by 45.1% to 64.5%, while maintaining a similar maximum reward level compared to DRL-based RM control with full learning from scratch. Specifically, the results also demonstrated that the proposed DRLCL-based RM outperformed the feedback-based RM due to its stronger predictive ability, faster response, and higher action precision.</p>","PeriodicalId":50381,"journal":{"name":"IET Intelligent Transport Systems","volume":null,"pages":null},"PeriodicalIF":2.3,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/itr2.12494","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139764389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Han Zhang, Hongbin Liang, Lei Wang, Yiting Yao, Bin Lin, Dongmei Zhao
Autonomous vehicles navigating urban roads require technology that combines low latency with high computing power. The limited resources of the vehicle itself compel it to offload task requirements to edge server (ES) for processing assistance. However, as the number of vehicles continues to increase, how edge servers reasonably allocate limited resources to autonomous vehicles becomes critical to the success of urban intelligent transportation services. This paper establishes an urban road scenario with multiple autonomous vehicles and an edge computing server and considers two main driving behaviour transition resource requests, namely car-following behaviour requests and lane-changing behaviour requests. Simultaneously, acknowledging that vehicles may encounter unforeseen traffic hazards when switching driving behaviours, a safety redundancy setting strategy is employed to allocate additional resources to the vehicle to ensure safety and model the vehicle resource allocation problem in the autonomous driving system. Double-deep Q-network (DDQN) is then used to solve this model and maximize the total system utility by comprehensively considering resource costs, system revenue, and autonomous vehicle safety. Finally, results from the simulation experiment indicate that the proposed dynamic resource allocation scheme, based on deep reinforcement learning for autonomous vehicles under edge computing, not only greatly improves the system's benefits and reduces processing delays compared to traditional greedy algorithms and value iteration, but also effectively ensures security.
{"title":"Joint resource allocation and security redundancy for autonomous driving based on deep reinforcement learning algorithm","authors":"Han Zhang, Hongbin Liang, Lei Wang, Yiting Yao, Bin Lin, Dongmei Zhao","doi":"10.1049/itr2.12489","DOIUrl":"10.1049/itr2.12489","url":null,"abstract":"<p>Autonomous vehicles navigating urban roads require technology that combines low latency with high computing power. The limited resources of the vehicle itself compel it to offload task requirements to edge server (ES) for processing assistance. However, as the number of vehicles continues to increase, how edge servers reasonably allocate limited resources to autonomous vehicles becomes critical to the success of urban intelligent transportation services. This paper establishes an urban road scenario with multiple autonomous vehicles and an edge computing server and considers two main driving behaviour transition resource requests, namely car-following behaviour requests and lane-changing behaviour requests. Simultaneously, acknowledging that vehicles may encounter unforeseen traffic hazards when switching driving behaviours, a safety redundancy setting strategy is employed to allocate additional resources to the vehicle to ensure safety and model the vehicle resource allocation problem in the autonomous driving system. Double-deep Q-network (DDQN) is then used to solve this model and maximize the total system utility by comprehensively considering resource costs, system revenue, and autonomous vehicle safety. Finally, results from the simulation experiment indicate that the proposed dynamic resource allocation scheme, based on deep reinforcement learning for autonomous vehicles under edge computing, not only greatly improves the system's benefits and reduces processing delays compared to traditional greedy algorithms and value iteration, but also effectively ensures security.</p>","PeriodicalId":50381,"journal":{"name":"IET Intelligent Transport Systems","volume":null,"pages":null},"PeriodicalIF":2.7,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/itr2.12489","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139764313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
On the cover: The cover image is based on the Research Article A multi-emission-driven efficient network design for green hub-and-spoke airline networks by Mengyuan Sun et al., https://doi.org/10.1049/itr2.12455.