An analysis of multi-agent reinforcement learning for decentralized inventory control systems

Abstract

Most solutions to the inventory management problem assume a centralization of information that is incompatible with organizational constraints in supply chain networks. The problem can be naturally decomposed into sub-problems, each associated with an independent entity, turning it into a multi-agent system. A decentralized solution to inventory management using multi-agent reinforcement learning (MARL) is proposed where each entity is controlled by an agent. Three multi-agent variations of the proximal policy optimization algorithm are investigated through simulations of different supply chain networks and levels of uncertainty. A framework is deployed, which relies on offline centralization during simulation-based policy identification but enables decentralization when the policies are deployed online to the real system. Results show that reducing information sharing constraints in training enables MARL to perform comparatively to a centralized learning-based solution when deployed, and to outperform a distributed model-based solution in most cases, whilst respecting the information constraints of the system.