Kai Peng;Jintao He;Jialu Guo;Yuan Liu;Jianwen He;Wei Liu;Menglan Hu
{"title":"Delay-Aware Optimization of Fine-Grained Microservice Deployment and Routing in Edge via Reinforcement Learning","authors":"Kai Peng;Jintao He;Jialu Guo;Yuan Liu;Jianwen He;Wei Liu;Menglan Hu","doi":"10.1109/TNSE.2024.3436616","DOIUrl":null,"url":null,"abstract":"Microservices have exerted a profound impact on the development of internet applications. Meanwhile, the growing number of mobile terminal user requests has made the communication between microservices extremely complex, significantly impacting the quality of user service experience in mobile edge computing. Therefore, the joint optimization of microservice deployment and request routing is necessary to alleviate server pressure and enhance overall performance of large-scaled MEC applications. However, most existing work studies the microservice deployment and request routing as two isolated problems and neglects the dependencies between microservices. This paper focuses on the data dependency relationship of request and multi-instance processing problem, and then formulate the joint problem of microservice deployment and request routing as an integer nonlinear program and queuing optimization model under complex constraints. To address this problem, we propose a fine-grained reinforcement learning-based algorithm named Reward Memory Shaping Deep Deterministic Policy Gradient (RMS \n<inline-formula><tex-math>$\\_$</tex-math></inline-formula>\n DDPG). Furthermore, we introduce the Long Short-Term Memory (LSTM) block into the actor network and critical network to make actions memorable. Finally, our experiments demonstrate that our algorithm is more superior in terms of delay target, load balancing and algorithm robustness compared with four baseline algorithms.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"11 6","pages":"6024-6037"},"PeriodicalIF":6.7000,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10631308/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
Microservices have exerted a profound impact on the development of internet applications. Meanwhile, the growing number of mobile terminal user requests has made the communication between microservices extremely complex, significantly impacting the quality of user service experience in mobile edge computing. Therefore, the joint optimization of microservice deployment and request routing is necessary to alleviate server pressure and enhance overall performance of large-scaled MEC applications. However, most existing work studies the microservice deployment and request routing as two isolated problems and neglects the dependencies between microservices. This paper focuses on the data dependency relationship of request and multi-instance processing problem, and then formulate the joint problem of microservice deployment and request routing as an integer nonlinear program and queuing optimization model under complex constraints. To address this problem, we propose a fine-grained reinforcement learning-based algorithm named Reward Memory Shaping Deep Deterministic Policy Gradient (RMS
$\_$
DDPG). Furthermore, we introduce the Long Short-Term Memory (LSTM) block into the actor network and critical network to make actions memorable. Finally, our experiments demonstrate that our algorithm is more superior in terms of delay target, load balancing and algorithm robustness compared with four baseline algorithms.
期刊介绍:
The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.