Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19

Abstract

Stockpiling and scheduling plans for medical supplies represent essential preventive and control measures in major public health events. In the face of major infectious diseases, such as the novel coronavirus disease (COVID-19), the outbreak trend and variability of disease strains are often unpredictable. Hence, it is necessary to optimally adjust the prevention and control dispatching strategy according to the circumstances and outbreak locations to maintain economic development while ensuring the human health survival, however, many models in this scenario seldom consider the dynamic material prediction and the measurement of multiple costs at the same time. Taking the COVID-19 scenario as a case study, we establish a deep reinforcement learning (DRL)-based medical supplies dispatching (MSD) model for major infectious diseases, considering the volatility of the COVID-19 situation and the discrepancy between medical material demand and supply due to the high infectiousness of the Omicron series strains. The present model has three main components: 1) First, for the dynamic medical material prediction problem in complex infectious disease scenarios, taking the lifted COVID-19 lockdown scenario as an example, the modified susceptible-exposed-infected-recovered (SEIR) model was utilized to analyze the spread of the COVID-19, understand its characteristics, and map out the related medical supplies demand; 2) Second, to break away from the previous premise of only considering supply-demand, this study adds scheduling rules and cost function that weighs health and economic costs. An epidemic dispatching optimization model (Epi_DispatchOptim) was established using the OpenAI Gym toolkit to form an environment structure with virus transmission space, and emergency MSD while considering both human health and economic costs. This architecture interprets the balance between the supply-demand of medical supplies and reflects the importance of MSD in the balanced development of health and economy under the spread of infectious diseases; 3) Finally, the MSD strategy under the balance of health and economic cost is explored in Epi_DispatchOptim using reinforcement learning (RL) and the evolutionary algorithm (EA). Experiments conducted on two datasets indicate that the RL and EA reduce economic as well as health costs compared to the original environmental strategies. The above study illustrates how to use epidemiological models to predict the demand for healthcare supplies as the premise of scheduling models, and use Epi_DispatchOptim to explore the dynamic MSD decisions under mortality and economic equilibrium. In Shanghai, China, the economic cost of the exploration strategy is reduced by 27.36–27.07B compared to static scheduling, and deaths are reduced by 126–150 in 150 day compared to the no-intervention scenario. By integrating knowledge of epidemiology, optimal decision making, and economics, Epi_DispatchOptim further constructs epidemiological models, cost functions, state-action spaces, and other modules to assist public health decision makers in adopting appropriate MSD strategies for major public health event.