Adaptive Tuning of Dynamic Matrix Control for Uncertain Industrial Systems With Deep Reinforcement Learning

Abstract

Dynamic matrix control (DMC) has been field-validated in many industrial practices, highlighting the critical importance of fine-tuning parameters for optimal performance. However, the tuning of well-performed parameters is challenging because the relationship between parameters and the performance of DMC is intricate to characterize for industrial systems with uncertainty. An adaptive tuning approach based on deep reinforcement learning (DRL) is proposed to optimize the performance of DMC for uncertain systems in this paper. The approach can online tune the horizons and weighting matrices of DMC in real time adaptive to the state and uncertainty of the systems. Compared with offline tuning approaches, the proposed approach does not need to tradeoff optimality for robustness. The proposed approach utilizes various state-of-the-art DRL algorithms, e.g., value-based and actor-critic-based, to develop online parameter tuning policies that can adapt to system uncertainty. A piecewise reward function is designed to improve the performance and stability of the agent. A novel predictor-switching criterion is developed to address the horizon inconsistency in the receding optimization process. The proposed approaches are validated by the moisture control task in industrial cigarette drying process. Note to Practitioners—This paper is motivated by the adaptive tuning problem of dynamic matrix control (DMC) in uncertain industrial systems. For other nonlinear industrial scenarios, practitioners should first design a nonlinear model predictive controller suitable for the controlled object. Then, they can refer to the proposed tuning algorithm to improve the controller performance. Specifically, regarding the setting of state and action sets, please refer to the technical details provided in this paper. The reward function can be flexibly set according to the needs of practitioners for the controller, e.g., improving the dynamic performance of the controller or saving controller energy consumption. The design processes of the tuning algorithms can refer to Algorithm 1 and Algorithm 2. There are two reasons why the proposed algorithm can be easily transferred to nonlinear industrial scenarios. First, the proposed algorithm does not restrict the model predictive controller or controlled object type. Second, the requirement for horizons or weighting matrices tuning widely exists in the practical applications of various model predictive algorithms.