Optimal ATM Cash Replenishment Planning in a Smart City using Deep Q-Network

ATMs are no longer just machines, these connected devices are smart, intelligent things in the Internet of Things (IoT). Access to cash for many in society is remaining essential during the current COVID-19 lock-down around the globe. A cash inventory management system is necessary to decide whether ATM should be replenished on each day of the week. In this paper, we study the real-time cash replenishment planning problem under outflow uncertainty where the fee of the security companies grows if the replenishment ends up falling on a weekends/holidays. Our model is based by the Double Deep Q-Network (DQN) algorithm which combines popular Q-learning with a deep neural network. The proposed method is used to control replenishment operation in order to minimize replenishment cost where the cash demand changes dynamically at each day. Experiment results show that our proposed method can work effectively on the real outflow time-series and it is able to reduce the ATM operational cost compared with the other state-of-the-art cash demand prediction schemes.