Learning-based optimal boundary control for parabolic distributed parameter system with actuator dynamics

Abstract

Considering actuator dynamics, we investigate a coupled system of parabolic partial differential equations (PDEs) and ordinary differential equations (ODEs), developing a data-driven boundary optimal controller based on iterative learning. Notably, the boundary input appears in the ODE-style actuator dynamics, making the boundary condition reduction quite difficult. Optimizing infinite-dimensional performance indexes in coupled Hilbert spaces is not a trivial task. This work is the first to solve the optimal control problem for a coupled PDE-ODE system with actuator dynamics under Neumann boundary conditions. We equivalently reformat the coupled PDE-ODE system into a system with homogeneous boundary conditions and then derive its singular perturbation form in an infinite-dimensional space. Subsequently, by constructing critic and actor networks, we design a novel model-free iterative learning optimal control algorithm where weighted residual techniques are used. The algorithm uses a rich set of arbitrary control policies rather than limiting to evaluation policies, enhancing the exploration capability of the learning algorithm and relaxing the requirement for persistent excitation conditions. Furthermore, the uniformly asymptotic stability of the closed-loop coupled system is demonstrated in the infinite-dimensional Hilbert space for each learning iteration, not only for the final one. Finally, the effectiveness of the proposed approach is verified by simulations on the diffusion-reaction process.