Distributed Q-learning Algorithm for Economic Dispatch of Smart Grid with Unknown Cost Functions

Abstract

In this paper, a distributed Q-learning algorithm is studied to solve the economic dispatch (ED) problem in smart grid. To tackle the ED problem in the presence of unknown cost functions, most of the existing methods are designed based on the global information of generation units, which would suffer from potential network attack. To conquer such limitation, the distributed Q-leaning algorithm consisting of distributed communication and reinforcement learning (RL) is proposed, where no global information is allowed to used, but information exchange among neighboring generation units can be available. In distributed Q-learning, each generation unit learns the local action-value function and collaborates to optimize the ED problem. Finally, the convergence and optimality of the proposed algorithm are proven, and the numerical simulation results demonstrate the effectiveness of the algorithm.