RL SolVeR Pro: Reinforcement Learning for Solving Vehicle Routing Problem

Abstract

Vehicle Routing Problem (VRP) is a well-known NP-hard combinatorial optimization problem at the heart of the transportation and logistics research. VRP can be exactly solved only for small instances of the problem with conventional methods. Traditionally this problem has been solved using heuristic methods for large instances even though there is no guarantee of optimality. Efficient solution adopted to VRP may lead to significant savings per year in large transportation and logistics systems. Much of the recent works using Reinforcement Learning are computationally intensive and face the three curse of dimensionality: explosions in state and action spaces and high stochasticity i.e., large number of possible next states for a given state action pair. Also, recent works on VRP don’t consider the realistic simulation settings of customer environments, stochastic elements and scalability aspects as they use only standard Solomon benchmark instances of at most 100 customers. In this work, Reinforcement Learning Solver for Vehicle Routing Problem (RL SolVeR Pro) is proposed wherein the optimal route learning problem is cast as a Markov Decision Process (MDP). The curse of dimensionality of RL is also overcome by using two-phase solver with geometric clustering. Also, realistic simulation for VRP was used to validate the effectiveness and applicability of the proposed RL SolVeR Pro under various conditions and constraints. Our simulation results suggest that our proposed method is able to obtain better or same level of results, compared to the two best-known heuristics: Clarke-Wright Savings and Sweep Heuristic. The proposed RL Solver can be applied to other variants of the VRP and has the potential to be applied more generally to other combinatorial optimization problems.