Human-in-the-loop control strategy for IoT-based smart thermostats with Deep Reinforcement Learning

Abstract

Thermostatic Radiator Valves (TRVs) are a widely used technology for regulating room heating in Europe countries. Smart TRVs can provide significant energy savings, often ranging from 20–40% compared to conventional heating systems. They use sensors and algorithms to learn user behavior and optimize heating schedules accordingly. They can often be easily retrofitted to existing heating systems, making them a practical option for enhancing energy efficiency in present buildings, especially in office buildings due to their highly dynamic operational patterns. This work presents a novel human-in-the-loop control strategy for Internet of Things (IoT)-based TRVs using Deep Reinforcement Learning (DRL). A key focus of this research is enhancing the adaptability of agents’ behavior by implementing a more generic and flexible Markov Decision Process (MDP) to promote policy generalization across diverse scenarios. The study explores the challenges of transferring control behaviors from simulation environments to real-world settings, examining the performance across different thermal zones and evaluating the integration flexibility of the control strategy within building systems. Real-world occupant behavior is incorporated, including dynamic comfort preferences and occupancy predictions, to better align thermostat operation with user preferences. Furthermore, this paper discusses the practical challenges encountered during implementation, including battery consumption of IoT devices, integration of occupancy detection and prediction systems, and maintenance requirements. By addressing these issues, the proposed control strategy seeks to improve the scalability and feasibility of IoT-based TRVs, thereby providing a viable solution for their widespread deployment in buildings.