Byzantine-Resilient Economical Operation Strategy Based on Federated Deep Reinforcement Learning for Multiple Electric Vehicle Charging Stations Considering Data Privacy

With the goal of low-carbon energy utilization, electric vehicles (EVs) and EV charging stations (EVCSs) are becoming increasingly popular. The economical operation strategy is always a primary concern for EVCSs, while users' behavior and operating data leakage problems in EVCSs have not been taken seriously. Herein, federated deep reinforcement learning, a privacy-preserving method, is applied to learn the optimal strategy for multiple EVCSs. However, it is prone to Byzantine attacks. It is urgent to achieve an economical operation strategy while preserving data privacy and defending against Byzantine attacks. Therefore, this paper proposes a Byzantine-resilient federated deep reinforcement learning (BR-FDRL) method to address these problems. First, the distributed EVCS data are utilized by the federated deep reinforcement learning to train an economical operation strategy while preserving privacy by only transmitting gradients. The sampling efficiency is enhanced by both federated learning and stochastically controlled stochastic gradient. Then, the Byzantine-resilient gradient filter (BRGF) designs two distance rules to keep malicious gradients out. The case study verifies the effectiveness of the proposed BRGF in resisting Byzantine attacks and the effectiveness of federated deep reinforcement learning in improving convergence speed and reward and preserving privacy. The resluts show that the BR-FDRL method minimizes the operation cost by an average of 35% compared with the rule-based method while meeting the state of charge demand as much as possible.