Reinforcement Learning based Data-driven Optimal Control Strategy for Systems with Disturbance

Abstract

This paper proposes a partially model-free optimal control strategy for a class of continuous-time systems in a data-driven way. Although a series of optimal control have achieving superior performance, the following challenges still exist: (i) The controller designed based on the nominal system is difficult to cope with sudden disturbances. (ii) Feedback control is highly dependent on system dynamics and generally requires full state information. A novel composite control method combining output feedback reinforcement learning and input-output disturbance observer for these two challenges is concluded in this paper. Firstly, an output feedback policy iteration (PI) algorithm is given to acquire the feedback gain iteratively. Simultaneously, the observer continuously provides estimates of the disturbance. System dynamic information and states information are not needed to be known in advance in our approach, thus offering a higher degree of robustness and practical implementation prospects. Finally, an example is given to show the effectiveness of the proposed controller.