Hawkes Process Multi-armed Bandits for Search and Rescue

Abstract

We propose a novel framework for integrating Hawkes processes with multi-armed bandit algorithms to solve spatio-temporal event forecasting and detection problems when data may be undersampled or spatially biased. In particular, we introduce an upper confidence bound algorithm using Bayesian spatial Hawkes process estimation for balancing the trade-off between exploiting geographic regions where data has been collected and exploring geographic regions where data is unobserved. We first validate our model using simulated data. We then apply it to the problem of disaster search and rescue using calls for service data from hurricane Harvey in 2017 and the problem of detection and clearance of improvised explosive devices (IEDs) using IED attack records in Iraq. Our model outperforms state-of-the-art baseline spatial MAB algorithms in terms of cumulative reward and several other ranking evaluation metrics.