An Actor-Critic Reinforcement Learning Control Approach for Discrete-Time Linear System with Uncertainty

Abstract

This paper is concerned with an adaptive optimal controller based an actor-critic architecture for solving discrete-time linear system with uncertainty. The actor-critic reinforcement learning progress is similar to the produce of dopamine in human brain and the mechanism which acts on the motoneuron, which dopamine enhances specific actions by reinforce the synaptic contact of the frontal lobe. As same as artificial intelligence (AI), it means the reward signal of dopamine in the neural network can be used to adjust weights in artificial neural which makes the system find the right way to solve the work. The actor-critic scheme is applied to solve the dynamic programming equation problem, using actor and critic neural networks (NNs) for solving optimal controller and optimal value function, respectively. The weights of actor and critic NNs are updated using policy gradient and recursive least squares temporal-difference learning (RLS-TD) scheme at each sampling instant. Finally, time and frequency domain simulations performed using a typical quarter-car suspension systems that an active suspension systems with the proposed control strategy is able to improve ride comfort significantly, compared with the conventional passive suspension systems.