Cost-Effective Power Delivery via Deep Reinforcement Learning-Based Dynamic Electric Vehicle Transportation

Abstract

Power delivery issues are increasingly evident in cyber-physical smart grid systems as energy transactions frequently overlook the physical constraints of distribution, leading to transmission congestion and compromising network security and reliability. This article presents a novel and cost-effective solution to power delivery challenges by utilizing electric vehicles (EVs) with dynamic transportation capabilities as free carriers. Unlike traditional approaches, a deep reinforcement learning (DRL)-based optimization framework is designed to effectively manage incomplete information in real-time. Our method first introduces an investment-free model that leverages existing EV routes to transport energy during congestion, operating in a “free-riding” transmission mode. This not only enhances network reliability but also curtails costs. Then, we develop a Markov decision process (MDP) for sequential decision-making of 24-h optimal control, aimed at minimizing operational losses including load shedding and battery degradation. To deal with the stochastic nature of energy requests and EV routes in the control problem, we employ a model-free DRL algorithm to tackle the challenge of incomplete information. An Actor-Critic network, combining value-based and policy-based approaches, helps discover approximately optimal strategies in a continuous action space. Finally, the simulation results numerically demonstrate the performance of the proposed method.