Multi-agent reinforcement learning for decentralized control of shared battery energy storage system in residential community

This article proposes a data-driven decentralized control scheme for a battery energy storage system, shared among residential PV households characterized by their respective uncontrollable demand and PV generation. The households are connected to the grid via the point of common coupling and are accordingly billed by the utility company. We firstly translate the decentralized control objective into a multi-agent reinforcement learning (MARL) problem by modelling the interaction between the agents and their environment as a Markov Game. Thereafter, we present the novel Distributed Subgradient $Q-$learners (DSQL) algorithm based on the localization of the Hyper$-Q$ function and the coordination among the learning agents connected via a communication network. The proposed algorithm holds merit in addressing the typical key-aspects of MARL algorithms, i.e., scalability, privacy and fairness. Finally, we perform numerical simulations by using real historical demand, PV generation and electricity tariff data and highlight the key advantages of the proposed algorithm w.r.t. the state-of-art, in terms of economic savings and key-performance indicators, such as peak-to-average ratio, valley-to-average ratio and root-mean-squared-deviation.