Hierarchical Optimization of Product System Using Reinforcement Learning

Abstract

As product systems become larger and more complex, their design space increases, making optimization more difficult. In the past, hierarchical optimization methods have been proposed to solve this problem. However, they are ineffective in difficult cases where subsystems are strongly coupled. Therefore, we focused on optimal solutions using reinforcement learning. However, for large-scale optimization problems, the learning space increases, and optimization becomes difficult. Therefore, we consider subsystem optimizers as agents and propose an algorithm that mitigates the disadvantages of reducing the learning space of reinforcement learning through negotiation between agents. Finally, the proposed method successfully reduces the number of evaluations to derive the optimal solution to less than 10% of the previous one, while maintaining the quality of the optimization solution, by increasing the efficiency of learning from 2.1% to 28.1% by reducing the learning space.