Solving Stochastic Orienteering Problems With Chance Constraints Using Monte Carlo Tree Search

Abstract

We present a new Monte Carlo Tree Search (MCTS) algorithm to solve the stochastic orienteering problem with chance constraints, i.e., a version of the problem where travel costs are random, and one is assigned a bound on the tolerable probability of exceeding the budget. The algorithm we present is online and anytime, i.e., it alternates planning and execution, and the quality of the solution it produces increases as the allowed computational time increases. Differently from most former MCTS algorithms, for each action available in a state the algorithm maintains estimates of both its value and the probability that its execution will eventually result in a violation of the chance constraint. Then, at action selection time, our proposed solution prunes away trajectories that are estimated to violate the failure probability. Extensive simulation results show that this approach can quickly produce high-quality solutions and is competitive with the optimal but time-consuming solution. Note to Practitioners—In many practical scenarios one is faced with multiobjective sequential decision making problems that can be solved through constrained optimization. If some of the parameters are known with uncertainty, the event “violating one of the constraints” becomes a random variable whose probability should be bound. As an application of this general problem formulation, in this paper we consider stochastic orienteering, a problem that finds applications when a robot is tasked with performing multiple tasks of varying utility while being subject to a bound on the traveled distance. Many problems in logistics, precision agriculture, and environmental monitoring, just to name a few, can be cast as instances of this optimization problem.