Guided Bayesian Optimization: Data-Efficient Controller Tuning With Digital Twin

Abstract

This article presents the guided Bayesian optimization (BO) algorithm as an efficient data-driven method for iteratively tuning closed-loop controller parameters using a digital twin of the system. The digital twin is built using closed-loop data acquired during standard BO iterations, and activated when the uncertainty in the Gaussian Process model of the optimization objective on the real system is high. We define a controller tuning framework independent of the controller or the plant structure. Our proposed methodology is model-free, making it suitable for nonlinear and unmodelled plants with measurement noise. The objective function consists of performance metrics modeled by Gaussian processes. We utilize the available information in the closed-loop system to progressively maintain a digital twin that guides the optimizer, improving the data efficiency of our method. Switching the digital twin on and off is triggered by our data-driven criteria related to the digital twin’s uncertainty estimations in the BO tuning framework. Effectively, it replaces much of the exploration of the real system with exploration performed on the digital twin. We analyze the properties of our method in simulation and demonstrate its performance on two real closed-loop systems with different plant and controller structures. The experimental results show that our method requires fewer experiments on the physical plant than Bayesian optimization to find the optimal controller parameters. Note to Practitioners—Industrial applications typically are difficult to model due to disturbances. Bayesian optimization is a data-efficient iterative tuning method for a black box system in which the performance can only be measured given the control parameters. Iterative measurements involve operational costs. We propose a guided Bayesian optimization method that uses all information flow in a system to define a simplified digital twin of the system using out-of-the-box methods. It is continuously updated with data from the system. We use the digital twin instead of the real system to perform experiments and to find optimal controller parameters while we monitor the uncertainty of the resulting predictions. When the uncertainty exceeds a tolerance threshold, the real system is measured, and the digital twin is updated. This results in fewer experiments on the real system only when needed. We then demonstrate the improved data efficiency of the guided Bayesian optimization on real-time linear and rotary motor hardware. These common industrial plants need to be controlled rigorously in a closed-loop system. Our method requires 57% and 46% fewer experiments on the hardware than Bayesian optimization to tune the control parameters of the linear and rotary motor systems. Our generic approach is not limited to the controller parameters but also can optimize the parameters of a manufacturing process.