VRCCS-AC: Reinforcement Learning for Service Migration in Vehicular Edge Computing Systems

Abstract

Existing service migration approaches provide minimal service delay for mobile vehicles (MVs) in vehicular edge computing (VEC) systems. Nonetheless, existing approaches focus more on formulating migration strategies rely on global information of the system, which may incur high signaling overhead and poor scalability. Furthermore, existing approaches are difficult to reuse previous migration strategies and necessitate significant interaction to adapt to new VEC scenarios. In this paper, we present a decentralized service migration approache base on Variational Recurrent and Critic-Coached Strategy reuse Actor-Critic (VRCCS-AC). First, a variational recurrent model (VRM) is introduced to efficiently obtain information from MV's local state through modeling VEC scenarios. An actor-critic enhances migration strategies by accessing both VEC scenario and VRM. Second, CCS leverages the critic-network to assess and select source service migration strategy. In every state, CCS selects the source strategy that exhibits the most significant one-step enhancement compared to the current target strategy, and develops a coached strategy. Then, the target strategy is regularized to imitate the coached strategy to facilitate effective strategy search and efficient strategy transfer. Experiments on the real-world datasets demonstrate that compared to the baselines, VRCCS-AC reduces latency by 10.11% $\sim$ 18.57% and can quickly transfer to new VEC scenarios.