高动态异构车载边缘计算的多目标任务卸载:一种高效的强化学习方法

IF 4.5 3区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS Computer Communications Pub Date : 2024-06-29 DOI:10.1016/j.comcom.2024.06.018
ZhiDong Huang, XiaoFei Wu, ShouBin Dong
{"title":"高动态异构车载边缘计算的多目标任务卸载:一种高效的强化学习方法","authors":"ZhiDong Huang,&nbsp;XiaoFei Wu,&nbsp;ShouBin Dong","doi":"10.1016/j.comcom.2024.06.018","DOIUrl":null,"url":null,"abstract":"<div><p>Vehicular Edge Computing (VEC) provides a flexible distributed computing paradigm for offloading computations to the vehicular network, which can effectively solve the problem of limited vehicle computing resources and meet the on-vehicle computing requests of users. However, the conflict of interest between vehicle users and service providers leads to the need to consider a variety of conflict optimization goals for computing offloading, and the dynamic nature of vehicle networks, such as vehicle mobility and time-varying network conditions, make the offloading effectiveness of vehicle computing requests and the adaptability to complex VEC scenarios challenging. To address these challenges, this paper proposes a multi-objective optimization model suitable for computational offloading of dynamic heterogeneous VEC networks. By formulating the dynamic multi-objective computational offloading problem as a multi-objective Markov Decision Process (MOMDP), this paper designs a novel multi-objective reinforcement learning algorithm EMOTO, which aims to minimize the average task execution delay and average vehicle energy consumption, and maximize the revenue of service providers. In this paper, a preference priority sampling module is proposed, and a model-augmented environment estimator is introduced to learn the environmental model for multi-objective optimization, so as to solve the problem that the agent is difficult to learn steadily caused by the highly dynamic change of VEC environment, thus to effectively realize the joint optimization of multiple objectives and improve the decision-making accuracy and efficiency of the algorithm. Experiments show that EMOTO has superior performance on multiple optimization objectives compared with advanced multi-objective reinforcement learning algorithms. In addition, the algorithm shows robustness when applied to different environmental settings and better adapting to highly dynamic environments, and balancing the conflict of interest between vehicle users and service providers.</p></div>","PeriodicalId":55224,"journal":{"name":"Computer Communications","volume":"225 ","pages":"Pages 27-43"},"PeriodicalIF":4.5000,"publicationDate":"2024-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multi-objective task offloading for highly dynamic heterogeneous Vehicular Edge Computing: An efficient reinforcement learning approach\",\"authors\":\"ZhiDong Huang,&nbsp;XiaoFei Wu,&nbsp;ShouBin Dong\",\"doi\":\"10.1016/j.comcom.2024.06.018\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Vehicular Edge Computing (VEC) provides a flexible distributed computing paradigm for offloading computations to the vehicular network, which can effectively solve the problem of limited vehicle computing resources and meet the on-vehicle computing requests of users. However, the conflict of interest between vehicle users and service providers leads to the need to consider a variety of conflict optimization goals for computing offloading, and the dynamic nature of vehicle networks, such as vehicle mobility and time-varying network conditions, make the offloading effectiveness of vehicle computing requests and the adaptability to complex VEC scenarios challenging. To address these challenges, this paper proposes a multi-objective optimization model suitable for computational offloading of dynamic heterogeneous VEC networks. By formulating the dynamic multi-objective computational offloading problem as a multi-objective Markov Decision Process (MOMDP), this paper designs a novel multi-objective reinforcement learning algorithm EMOTO, which aims to minimize the average task execution delay and average vehicle energy consumption, and maximize the revenue of service providers. In this paper, a preference priority sampling module is proposed, and a model-augmented environment estimator is introduced to learn the environmental model for multi-objective optimization, so as to solve the problem that the agent is difficult to learn steadily caused by the highly dynamic change of VEC environment, thus to effectively realize the joint optimization of multiple objectives and improve the decision-making accuracy and efficiency of the algorithm. Experiments show that EMOTO has superior performance on multiple optimization objectives compared with advanced multi-objective reinforcement learning algorithms. In addition, the algorithm shows robustness when applied to different environmental settings and better adapting to highly dynamic environments, and balancing the conflict of interest between vehicle users and service providers.</p></div>\",\"PeriodicalId\":55224,\"journal\":{\"name\":\"Computer Communications\",\"volume\":\"225 \",\"pages\":\"Pages 27-43\"},\"PeriodicalIF\":4.5000,\"publicationDate\":\"2024-06-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computer Communications\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0140366424002287\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0140366424002287","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

车载边缘计算(Vehicular Edge Computing,VEC)为将计算卸载到车载网络提供了一种灵活的分布式计算范式,可以有效解决车载计算资源有限的问题,满足用户的车载计算需求。然而,车辆用户和服务提供商之间的利益冲突导致计算卸载需要考虑各种冲突优化目标,而车辆网络的动态特性,如车辆移动性和网络条件的时变性,使得车辆计算请求的卸载有效性和对复杂 VEC 场景的适应性面临挑战。针对这些挑战,本文提出了一种适用于动态异构 VEC 网络计算卸载的多目标优化模型。通过将动态多目标计算卸载问题表述为多目标马尔可夫决策过程(MOMDP),本文设计了一种新颖的多目标强化学习算法 EMOTO,其目标是使平均任务执行延迟和平均车辆能耗最小化,并使服务提供商的收益最大化。本文提出了偏好优先级抽样模块,并引入了模型增强环境估计器来学习多目标优化的环境模型,从而解决了 VEC 环境高度动态变化导致的代理难以稳定学习的问题,有效地实现了多目标的联合优化,提高了算法的决策精度和效率。实验表明,与先进的多目标强化学习算法相比,EMOTO 在多优化目标上具有更优越的性能。此外,该算法在应用于不同环境设置时表现出鲁棒性,能更好地适应高动态环境,并平衡车辆用户与服务提供商之间的利益冲突。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Multi-objective task offloading for highly dynamic heterogeneous Vehicular Edge Computing: An efficient reinforcement learning approach

Vehicular Edge Computing (VEC) provides a flexible distributed computing paradigm for offloading computations to the vehicular network, which can effectively solve the problem of limited vehicle computing resources and meet the on-vehicle computing requests of users. However, the conflict of interest between vehicle users and service providers leads to the need to consider a variety of conflict optimization goals for computing offloading, and the dynamic nature of vehicle networks, such as vehicle mobility and time-varying network conditions, make the offloading effectiveness of vehicle computing requests and the adaptability to complex VEC scenarios challenging. To address these challenges, this paper proposes a multi-objective optimization model suitable for computational offloading of dynamic heterogeneous VEC networks. By formulating the dynamic multi-objective computational offloading problem as a multi-objective Markov Decision Process (MOMDP), this paper designs a novel multi-objective reinforcement learning algorithm EMOTO, which aims to minimize the average task execution delay and average vehicle energy consumption, and maximize the revenue of service providers. In this paper, a preference priority sampling module is proposed, and a model-augmented environment estimator is introduced to learn the environmental model for multi-objective optimization, so as to solve the problem that the agent is difficult to learn steadily caused by the highly dynamic change of VEC environment, thus to effectively realize the joint optimization of multiple objectives and improve the decision-making accuracy and efficiency of the algorithm. Experiments show that EMOTO has superior performance on multiple optimization objectives compared with advanced multi-objective reinforcement learning algorithms. In addition, the algorithm shows robustness when applied to different environmental settings and better adapting to highly dynamic environments, and balancing the conflict of interest between vehicle users and service providers.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Computer Communications
Computer Communications 工程技术-电信学
CiteScore
14.10
自引率
5.00%
发文量
397
审稿时长
66 days
期刊介绍: Computer and Communications networks are key infrastructures of the information society with high socio-economic value as they contribute to the correct operations of many critical services (from healthcare to finance and transportation). Internet is the core of today''s computer-communication infrastructures. This has transformed the Internet, from a robust network for data transfer between computers, to a global, content-rich, communication and information system where contents are increasingly generated by the users, and distributed according to human social relations. Next-generation network technologies, architectures and protocols are therefore required to overcome the limitations of the legacy Internet and add new capabilities and services. The future Internet should be ubiquitous, secure, resilient, and closer to human communication paradigms. Computer Communications is a peer-reviewed international journal that publishes high-quality scientific articles (both theory and practice) and survey papers covering all aspects of future computer communication networks (on all layers, except the physical layer), with a special attention to the evolution of the Internet architecture, protocols, services, and applications.
期刊最新文献
Editorial Board A deep dive into cybersecurity solutions for AI-driven IoT-enabled smart cities in advanced communication networks The pupil outdoes the master: Imperfect demonstration-assisted trust region jamming policy optimization against frequency-hopping spread spectrum High-performance BFT consensus for Metaverse through block linking and shortcut loop Automating 5G network slice management for industrial applications
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1