A Deep Reinforcement Learning Method Solving Bilevel Optimization for Product Design Considering Remanufacturing

Abstract

Green laws make original equipment manufacturers responsible for full product lifecycle management, emphasizing remanufacturing. Research shows that remanufacturing technologies are complex and costly. Without product designs tailored for remanufacturing, achieving efficiency becomes a significant challenge. Therefore, it is imperative to consider remanufacturing during the initial product design stage. Existing literature primarily proposes either integrated or two-stage optimization methods for the decision-making of manufacturers and remanufacturers. However, they fail to describe the tradeoffs between the decisions of the two stakeholders. This article proposes a leader–follower interactive decision-making framework based on a Stackelberg game to explore the interaction between product design and remanufacturing and construct a bilevel interactive optimization (BIO) model. To solve it, we further develop a novel bilevel deep reinforcement learning framework, which can be applied to general BIO problems, particularly with multidimensional discrete decision variables and complex model constraints. We validate the proposed model and algorithm through case studies on laptops and electric vehicles, supported by comprehensive comparative experiments. Our results show that the product design considering the remanufacturing process improves manufacturers' utility per unit cost while reducing remanufacturers' costs.