A scalable multi-agent deep reinforcement learning in thermoforming: An experimental evaluation of thermal control by infrared camera-based feedback

This manuscript presents the development of multi-agent Deep Reinforcement Learning (DRL) for radiation thermal control in thermoforming processes involving multiple heaters. The complexity of such control systems is characterized by significant action and state spaces, where the actions of all actuators collectively influence the system's output. This complexity introduces substantial challenges regarding the computational demands for offline training of learning-based algorithms and the online computational costs associated with a real-world controller deployment. The study presents a novel approach to training an adaptive and robust DRL agent system that can control a single heating element on the thermoplastic sheet while dynamically considering interactive effects from nearby heaters. Results demonstrated that upon deploying the pre-trained agent for each heater within the heater bank, the group of agents could then regulate the temperature of the sheet to any physically feasible output temperature profile. In contrast to the conventional DRL approach, where a single agent manages all heaters, the multi-agent DRL method boasted that an offline training process was 110 times faster, coupled with an 8 times reduction in the final error margin on the simulator. The experimental data, conducted on a laboratory-scale setup, confirmed the performance of the proposed model, with a final absolute error under 4 ${}^{°}C$. Regardless of the number of heaters, the multi-agent DRL approach exhibited accurate and robust performance. Its advantage was that it incurred no significant offline and online computational burden when the number of heating elements increased, deemed a promising notion for industrial-scale applications.