A Combination Feature-Based Reinforcement Learning Approach via Mathematical Optimization

Abstract

Reinforcement learning is a promising method for solving decision problems, and its potential has been increasingly recognized for large-scale combinatorial optimization problems in recent years. However, the existing studies on reinforcement learning for cutting stock problems mostly rely on sequence-to-sequence or graph neural network approaches that use the learned experience to make decisions while neglecting the combination features of cutting stock problems. In this paper, we propose a novel reinforcement learning framework for cutting stock problems that integrate integer programming and monotone comparative statics to construct a Markov decision process with a high-quality action space. We start by constructing a new Markov decision process that considers the diagonal structure of the integer programming model for combinatorial optimization problems, and then use column generation to obtain each action by combining multiple decision variables. Furthermore, we design a bipartite graph and related bipartite graph convolutional network to find the solutions. The results show that the proposed reinforcement learning framework provides a high-quality action space, and the designed bipartite graph convolutional network can effectively select the best actions from the action set.Note to Practitioners—This article was motivated by the cutting stock problems that exist in various industrial scenarios such as the wood, steel, paper, and glass industries. We improve the reinforcement learning for the cutting stock problem that can be adopted in industrial scenarios, which can increase the profile and reduce the production cost of industrial enterprises. Our improvement can also be referred to when solving other combinatorial optimization problems that can promote making decisions in industrial production.