Joint optimization for service-caching, computation-offloading, and UAVs flight trajectories over rechargeable UAV-aided MEC using hierarchical multi-agent deep reinforcement learning

IF 5.8 2区 计算机科学 Q1 TELECOMMUNICATIONS Vehicular Communications Pub Date : 2024-09-11 DOI:10.1016/j.vehcom.2024.100844
Zhian Chen, Fei Wang, Jiaojie Wang
{"title":"Joint optimization for service-caching, computation-offloading, and UAVs flight trajectories over rechargeable UAV-aided MEC using hierarchical multi-agent deep reinforcement learning","authors":"Zhian Chen,&nbsp;Fei Wang,&nbsp;Jiaojie Wang","doi":"10.1016/j.vehcom.2024.100844","DOIUrl":null,"url":null,"abstract":"<div><p>Due to the high mobility, high chance of line-of-sight (LoS) transmission, and flexible deployment, unmanned aerial vehicles (UAVs) have been used as mobile edge computing (MEC) servers to provide ubiquitous computation services to mobile users (MUs). However, the limited energy storage, caching capacity, and computation resources of UAVs bring new challenges for UAV-aided MEC, e.g., how to recharge UAVs and how to jointly optimize service-caching, computation-offloading, and UAVs flight trajectories. To overcome the above-mentioned difficulties, in this paper we study the joint optimization for service-caching, computation-offloading, and UAVs flight trajectories for UAV-aided MEC, where multiple rechargeable UAVs cooperatively provide MEC services to a number of MUs. First, we formulate an energy minimization problem to minimize all MUs' energy consumptions by taking into account the mobility of MUs and the energy replenishment of UAVs. Then, using the <em>hierarchical multi-agent deep reinforcement learning</em> (<u>HMDRL</u>), we develop a two-timescale based joint <u>s</u>ervice-<u>c</u>aching, <u>c</u>omputation-<u>o</u>ffloading, and UAVs <u>f</u>light <u>t</u>rajectories scheme, called <em>HMDRL-Based SCOFT</em>. Using HMDRL-Based SCOFT, we derive UAVs' service-caching policies in each time frame, and then derive UAVs flight trajectories and MUs' computation-offloading in each time slot. Finally, we validate and evaluate the performances of our proposed HMDRL-Based SCOFT scheme through extensive simulations, which show that our developed scheme outperforms the other baseline schemes to converge faster and greatly reduce MUs' energy consumptions.</p></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":"50 ","pages":"Article 100844"},"PeriodicalIF":5.8000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209624001190","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0

Abstract

Due to the high mobility, high chance of line-of-sight (LoS) transmission, and flexible deployment, unmanned aerial vehicles (UAVs) have been used as mobile edge computing (MEC) servers to provide ubiquitous computation services to mobile users (MUs). However, the limited energy storage, caching capacity, and computation resources of UAVs bring new challenges for UAV-aided MEC, e.g., how to recharge UAVs and how to jointly optimize service-caching, computation-offloading, and UAVs flight trajectories. To overcome the above-mentioned difficulties, in this paper we study the joint optimization for service-caching, computation-offloading, and UAVs flight trajectories for UAV-aided MEC, where multiple rechargeable UAVs cooperatively provide MEC services to a number of MUs. First, we formulate an energy minimization problem to minimize all MUs' energy consumptions by taking into account the mobility of MUs and the energy replenishment of UAVs. Then, using the hierarchical multi-agent deep reinforcement learning (HMDRL), we develop a two-timescale based joint service-caching, computation-offloading, and UAVs flight trajectories scheme, called HMDRL-Based SCOFT. Using HMDRL-Based SCOFT, we derive UAVs' service-caching policies in each time frame, and then derive UAVs flight trajectories and MUs' computation-offloading in each time slot. Finally, we validate and evaluate the performances of our proposed HMDRL-Based SCOFT scheme through extensive simulations, which show that our developed scheme outperforms the other baseline schemes to converge faster and greatly reduce MUs' energy consumptions.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用分层多代理深度强化学习,在可充电无人机辅助 MEC 上联合优化服务缓存、计算卸载和无人机飞行轨迹
由于具有高机动性、高视距(LoS)传输几率和灵活部署等特点,无人机(UAV)已被用作移动边缘计算(MEC)服务器,为移动用户(MU)提供无处不在的计算服务。然而,无人机有限的储能、缓存能力和计算资源为无人机辅助移动计算(MEC)带来了新的挑战,例如,如何为无人机充电,如何共同优化服务缓存、计算卸载和无人机飞行轨迹。为了克服上述困难,本文研究了无人机辅助 MEC 的服务缓存、计算卸载和无人机飞行轨迹的联合优化问题。首先,我们提出了一个能量最小化问题,通过考虑 MU 的移动性和无人机的能量补充,最小化所有 MU 的能量消耗。然后,利用分层多代理深度强化学习(HMDRL),我们开发了一种基于双时间尺度的联合服务缓存、计算卸载和无人机飞行轨迹方案,称为基于 HMDRL 的 SCOFT。利用基于 HMDRL 的 SCOFT,我们得出了无人机在每个时间帧中的服务缓存策略,然后得出了无人机在每个时隙中的飞行轨迹和 MU 的计算卸载。最后,我们通过大量仿真验证和评估了我们提出的基于 HMDRL 的 SCOFT 方案的性能,结果表明我们开发的方案优于其他基线方案,收敛速度更快,大大降低了 MU 的能耗。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Vehicular Communications
Vehicular Communications Engineering-Electrical and Electronic Engineering
CiteScore
12.70
自引率
10.40%
发文量
88
审稿时长
62 days
期刊介绍: Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier. The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications: Vehicle to vehicle and vehicle to infrastructure communications Channel modelling, modulating and coding Congestion Control and scalability issues Protocol design, testing and verification Routing in vehicular networks Security issues and countermeasures Deployment and field testing Reducing energy consumption and enhancing safety of vehicles Wireless in–car networks Data collection and dissemination methods Mobility and handover issues Safety and driver assistance applications UAV Underwater communications Autonomous cooperative driving Social networks Internet of vehicles Standardization of protocols.
期刊最新文献
Decentralized multi-hop data processing in UAV networks using MARL Prediction-based data collection of UAV-assisted Maritime Internet of Things Hybrid mutual authentication for vehicle-to-infrastructure communication without the coverage of roadside units Hierarchical federated deep reinforcement learning based joint communication and computation for UAV situation awareness Volunteer vehicle assisted dependent task offloading based on ant colony optimization algorithm in vehicular edge computing
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1