A rollout heuristic-reinforcement learning hybrid algorithm for disassembly sequence planning with uncertain depreciation condition and diversified recovering strategies

Abstract

Disassembly is one of the crucial aspects of green manufacturing. For the end-of-life products, an effective disassembly sequence planning method can enhance recovery value and mitigate the negative consequences of resource depletion and waste generation. However, both the uncertainty of product depreciation condition and the NP-hard characteristics (including the determination of disassembly sequences and the selection of recovering strategies of subassemblies) of the disassembly sequence planning results in difficulties to determine the optimal/near-optimal disassembly solutions. To address these challenges, this work establishes an extended Petri net that considers diversified recovering strategies of each subassembly caused by uncertain product depreciation condition. Then, a rollout heuristic-reinforcement learning hybrid algorithm that integrates a rollout decision rule into the reinforcement learning procedure is proposed to rapidly find the high-quality disassembly solutions based on the extended Petri net, in which the uncertainty of disassembly information is tackled by training disassembly samples and the global exploration capability of the learning procedure is significantly improved by using the rollout decision rule. Finally, three products with different complexities and sizes are used to verify the performance of the proposed algorithm, and the experimental results indicate that our proposed rollout heuristic-reinforcement learning hybrid algorithm can efficiently compute the high-quality disassembly sequences under various disassembly environments.