Modeling of Ambient Comfort Affect Reward based on multi-agents in cloud interconnection environment for developing the sustainable home controller

Abstract

The paper presents a research based on a vision of a multi-agent model working for the ambient comfort measurement and environment control system. Such means are used for developing the Smarter Eco-Social Laboratory (SrESL). The human Ambient Comfort Affect Reward (ACAR) index is proposed for development of the Reinforcement Learning Based Ambient Comfort Controller (RL-ACC) for experiments using equipment of SrESL. The ACAR index is recognized as dependent on human physiological parameters, such as the temperature, the electrocardiogram (ECG) and the electro-dermal activity (EDA). The fuzzy logic is used to approximate the ACAR index function by defining two fuzzy inference systems: the Arousal-Valence System, and the Ambient Comfort Affect Reward (ACAR) System. The goal of the RL-ACC is to find such the environmental state characteristics that create an optimal comfort for people affected by this environment. The Radial Basis Neural Network is used as the main component of the RL-ACC to performing of two roles: the policy structure, known as the Actor, used to select actions, and the estimated value function, known as the Critic that criticizes the actions made by the Actor. The Actor which manages Critic processes was used as a value function approximation of the continuous learning tasks of the RL-ACC and presented in this paper.