Federated Accelerated Deep Reinforcement Learning for Multi-Zone HVAC Control in Commercial Buildings

Abstract

Deep Reinforcement Learning (DRL) has demonstrated a promising prospect in optimizing Heating, Ventilation, and Air Conditioning (HVAC) systems to minimize energy costs and improve thermal comfort without requiring the knowledge of building thermal dynamic models. However, DRL training can be difficult to converge due to its long exploration, especially in multi-agent scenarios. To address these concerns, we propose a novel Federated Accelerated Multi-Agent DRL (FA-MADRL) algorithm for HVAC control in commercial buildings with multi-zone offices. To be specific, we reformulate the optimal control problem of indoor temperature, CO2 concentration and humidity in multi-zone HVAC as a Markov Decision Process (MDP). Then, we establish a MADRL framework for HVAC system control and utilize a federated learning (FL) mechanism to accelerate the convergence during real-time deployment. Experimental studies have been carried out on a TRNSYS-based commercial building HVAC system with multiple zones. The results demonstrate the superiority of our proposed algorithm, with improved convergence speed, reduced energy consumption, and satisfied thermal comfort.