Joint optimization for service-caching, computation-offloading, and UAVs flight trajectories over rechargeable UAV-aided MEC using hierarchical multi-agent deep reinforcement learning
{"title":"Joint optimization for service-caching, computation-offloading, and UAVs flight trajectories over rechargeable UAV-aided MEC using hierarchical multi-agent deep reinforcement learning","authors":"Zhian Chen, Fei Wang, Jiaojie Wang","doi":"10.1016/j.vehcom.2024.100844","DOIUrl":null,"url":null,"abstract":"<div><p>Due to the high mobility, high chance of line-of-sight (LoS) transmission, and flexible deployment, unmanned aerial vehicles (UAVs) have been used as mobile edge computing (MEC) servers to provide ubiquitous computation services to mobile users (MUs). However, the limited energy storage, caching capacity, and computation resources of UAVs bring new challenges for UAV-aided MEC, e.g., how to recharge UAVs and how to jointly optimize service-caching, computation-offloading, and UAVs flight trajectories. To overcome the above-mentioned difficulties, in this paper we study the joint optimization for service-caching, computation-offloading, and UAVs flight trajectories for UAV-aided MEC, where multiple rechargeable UAVs cooperatively provide MEC services to a number of MUs. First, we formulate an energy minimization problem to minimize all MUs' energy consumptions by taking into account the mobility of MUs and the energy replenishment of UAVs. Then, using the <em>hierarchical multi-agent deep reinforcement learning</em> (<u>HMDRL</u>), we develop a two-timescale based joint <u>s</u>ervice-<u>c</u>aching, <u>c</u>omputation-<u>o</u>ffloading, and UAVs <u>f</u>light <u>t</u>rajectories scheme, called <em>HMDRL-Based SCOFT</em>. Using HMDRL-Based SCOFT, we derive UAVs' service-caching policies in each time frame, and then derive UAVs flight trajectories and MUs' computation-offloading in each time slot. Finally, we validate and evaluate the performances of our proposed HMDRL-Based SCOFT scheme through extensive simulations, which show that our developed scheme outperforms the other baseline schemes to converge faster and greatly reduce MUs' energy consumptions.</p></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":"50 ","pages":"Article 100844"},"PeriodicalIF":5.8000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209624001190","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Due to the high mobility, high chance of line-of-sight (LoS) transmission, and flexible deployment, unmanned aerial vehicles (UAVs) have been used as mobile edge computing (MEC) servers to provide ubiquitous computation services to mobile users (MUs). However, the limited energy storage, caching capacity, and computation resources of UAVs bring new challenges for UAV-aided MEC, e.g., how to recharge UAVs and how to jointly optimize service-caching, computation-offloading, and UAVs flight trajectories. To overcome the above-mentioned difficulties, in this paper we study the joint optimization for service-caching, computation-offloading, and UAVs flight trajectories for UAV-aided MEC, where multiple rechargeable UAVs cooperatively provide MEC services to a number of MUs. First, we formulate an energy minimization problem to minimize all MUs' energy consumptions by taking into account the mobility of MUs and the energy replenishment of UAVs. Then, using the hierarchical multi-agent deep reinforcement learning (HMDRL), we develop a two-timescale based joint service-caching, computation-offloading, and UAVs flight trajectories scheme, called HMDRL-Based SCOFT. Using HMDRL-Based SCOFT, we derive UAVs' service-caching policies in each time frame, and then derive UAVs flight trajectories and MUs' computation-offloading in each time slot. Finally, we validate and evaluate the performances of our proposed HMDRL-Based SCOFT scheme through extensive simulations, which show that our developed scheme outperforms the other baseline schemes to converge faster and greatly reduce MUs' energy consumptions.
期刊介绍:
Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier.
The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications:
Vehicle to vehicle and vehicle to infrastructure communications
Channel modelling, modulating and coding
Congestion Control and scalability issues
Protocol design, testing and verification
Routing in vehicular networks
Security issues and countermeasures
Deployment and field testing
Reducing energy consumption and enhancing safety of vehicles
Wireless in–car networks
Data collection and dissemination methods
Mobility and handover issues
Safety and driver assistance applications
UAV
Underwater communications
Autonomous cooperative driving
Social networks
Internet of vehicles
Standardization of protocols.