A novel multi-agent reinforcement learning framework for robust exception handling of manufacturing service collaboration based on asymmetric information

Abstract

Industrial internet platforms enable users to efficiently fulfill their customized needs through the sequential execution of a manufacturing service collaborative chain (MSCC) consisting of networked enterprises. However, various dynamic uncertainties (e.g., equipment failure, emergency order insertion, product quality deterioration) may interrupt the execution of the MSCC, resulting in processing overruns and reduced user willingness to customize. To enhance the ability of MSCC to respond to exception events (namely robustness), the asymmetric informative multi-agent reinforcement learning (AIMARL) method is proposed. AIMARL will re-select the appropriate manufacturing service for the unexecuted subtasks in the event of an MSCC exception. First, the method gives a definition way of MSCC robustness labels from the perspective of the platform and networked enterprises. Subsequently, the asymmetric cascade state and data-rule-driven asymmetric reward are designed based on the characteristics of unidirectional asymmetric information transmission in the sequential execution of the MSCC. Meanwhile, in order to fully utilize the graph features of the MSCC and extract the complex relationships between services, graph convolutional networks are embedded in both the asymmetric cascade state and data-rule-driven asymmetric reward. Experimental results demonstrate that AIMARL outperforms the other four multi-agent reinforcement learning methods for the problem. In addition, AIMARL is able to cope with dynamic uncertainties with better robustness than the anomaly handling methods used in the platform.