{"title":"A scalable multi-agent deep reinforcement learning in thermoforming: An experimental evaluation of thermal control by infrared camera-based feedback","authors":"","doi":"10.1016/j.jmapro.2024.09.019","DOIUrl":null,"url":null,"abstract":"<div><p>This manuscript presents the development of multi-agent Deep Reinforcement Learning (DRL) for radiation thermal control in thermoforming processes involving multiple heaters. The complexity of such control systems is characterized by significant action and state spaces, where the actions of all actuators collectively influence the system's output. This complexity introduces substantial challenges regarding the computational demands for offline training of learning-based algorithms and the online computational costs associated with a real-world controller deployment. The study presents a novel approach to training an adaptive and robust DRL agent system that can control a single heating element on the thermoplastic sheet while dynamically considering interactive effects from nearby heaters. Results demonstrated that upon deploying the pre-trained agent for each heater within the heater bank, the group of agents could then regulate the temperature of the sheet to any physically feasible output temperature profile. In contrast to the conventional DRL approach, where a single agent manages all heaters, the multi-agent DRL method boasted that an offline training process was 110 times faster, coupled with an 8 times reduction in the final error margin on the simulator. The experimental data, conducted on a laboratory-scale setup, confirmed the performance of the proposed model, with a final absolute error under 4 <span><math><msup><mrow></mrow><mo>°</mo></msup><mi>C</mi></math></span>. Regardless of the number of heaters, the multi-agent DRL approach exhibited accurate and robust performance. Its advantage was that it incurred no significant offline and online computational burden when the number of heating elements increased, deemed a promising notion for industrial-scale applications.</p></div>","PeriodicalId":16148,"journal":{"name":"Journal of Manufacturing Processes","volume":null,"pages":null},"PeriodicalIF":6.1000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Manufacturing Processes","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1526612524009241","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MANUFACTURING","Score":null,"Total":0}
引用次数: 0
Abstract
This manuscript presents the development of multi-agent Deep Reinforcement Learning (DRL) for radiation thermal control in thermoforming processes involving multiple heaters. The complexity of such control systems is characterized by significant action and state spaces, where the actions of all actuators collectively influence the system's output. This complexity introduces substantial challenges regarding the computational demands for offline training of learning-based algorithms and the online computational costs associated with a real-world controller deployment. The study presents a novel approach to training an adaptive and robust DRL agent system that can control a single heating element on the thermoplastic sheet while dynamically considering interactive effects from nearby heaters. Results demonstrated that upon deploying the pre-trained agent for each heater within the heater bank, the group of agents could then regulate the temperature of the sheet to any physically feasible output temperature profile. In contrast to the conventional DRL approach, where a single agent manages all heaters, the multi-agent DRL method boasted that an offline training process was 110 times faster, coupled with an 8 times reduction in the final error margin on the simulator. The experimental data, conducted on a laboratory-scale setup, confirmed the performance of the proposed model, with a final absolute error under 4 . Regardless of the number of heaters, the multi-agent DRL approach exhibited accurate and robust performance. Its advantage was that it incurred no significant offline and online computational burden when the number of heating elements increased, deemed a promising notion for industrial-scale applications.
期刊介绍:
The aim of the Journal of Manufacturing Processes (JMP) is to exchange current and future directions of manufacturing processes research, development and implementation, and to publish archival scholarly literature with a view to advancing state-of-the-art manufacturing processes and encouraging innovation for developing new and efficient processes. The journal will also publish from other research communities for rapid communication of innovative new concepts. Special-topic issues on emerging technologies and invited papers will also be published.