Incremental value iteration for optimal output regulation of linear systems with unknown exosystems

Abstract

This paper addresses the optimal output regulation problem for discrete-time linear systems with completely unknown dynamics and unmeasurable exosystem states. The primary objective is to design incremental dataset-based value iteration (VI) reinforcement learning algorithms to derive both state feedback and output feedback controllers. In the context of data-driven optimal control, existing approaches typically require either the exosystem state to be measurable or the design of an autonomous system to reconstruct it. In contrast, this work proposes an incremental dataset-based VI algorithm, which eliminates the need for exosystem state measurement or reconstruction. Additionally, the proposed method allows for the selection of an arbitrary initial admissible control policy, thereby overcoming the challenge of requiring an initial admissible control in policy iteration algorithms. Furthermore, the system state is reconstructed using the incremental dataset, and an optimal output feedback controller is developed based on the proposed VI algorithm. The theoretical convergence of the dataset-based incremental VI algorithm is rigorously analyzed, and comprehensive simulations are conducted to validate its effectiveness.