Reinforcement learning approach for outbound container stacking in container terminals

Abstract

The container stacking problem for outbound containers is a major planning task under yard operations in a container terminal. It is important to minimize the expected number of rehandling operations, which further helps maintain the productivity of yard operations and improve the efficiency of container terminals. In this paper, we propose a reinforcement learning based approach to determine the storage location of the arriving outbound containers. A reinforcement learning approach is developed to identify the appropriate storage location, aiming to minimize the expected number of rehandling operations during the loading operation. Furthermore, we developed suitable strategies related to reinforcement learning to determine the storage location by training the model using a sufficient number of episodes. Numerical experiments were conducted to compare the proposed model with existing algorithms using real-life container terminal data. The experimental results indicate that the proposed model is robust to uncertain environments, supports real-time decisions, and minimizes the expected number of rehandling operations.