Q-learning based scheduling method for continuous pickling process of titanium strips

Abstract

This article addresses the energy consumption optimization problems of the pickling process for titanium strip manufacturing. The hybrid flow shop scheduling schemes for the pickling process of titanium strips are designed, and a novel shop scheduling method based on reinforcement learning is proposed for the pickling process of titanium strips. In the scheduling scheme, the pickling chemical treatment process of titanium strips are described as an asymmetric hybrid flow shop scheduling problem (AHFSP), and a mathematical model containing a temperature structure is established with the optimization objectives of minimizing pickling time and energy consumption. Based on the proposed scheduling scheme, a novel shop scheduling method based on reinforcement learning for the titanium strip pickling process is proposed. First, a mixed integer linear programing model for the mixed flow shop scheduling problem is established. Second, the flow shop scheduling problem with sequential energy consumption decisions is approximated as an asymmetric traveling sales-man problem (ATSP). Finally, the ATSP is described as a Markov decision processes (MDP), and a Q-learning based scheduling method for titanium strip pickling shops is proposed. Finally, the effectiveness of the proposed method is verified by examples, and the scheduling scheme can reduce the energy consumption by 16.61% on average while maintaining the schedule, which improves the productivity and economic efficiency.