A multi-task deep reinforcement learning approach to real-time railway train rescheduling

In high-speed railway systems, unexpected disruptions can result in delays of trains, significantly affecting the quality of service for passengers. Train Timetable Rescheduling (TTR) is a crucial task in the daily operation of high-speed railways to maintain punctuality and efficiency in the face of such unforeseen disruptions. Most existing studies on TTR are based on integer programming (IP) techniques and are required to solve IP models repetitively in case of disruptions, which however may be very time-consuming and greatly limit their usefulness in practice. Our study first proposes a multi-task deep reinforcement learning (MDRL) approach for TTR. Our MDRL is constructed and trained offline with a large number of historical disruptive events, enabling to generate TTR decisions in real-time for different disruption cases. Specifically, we transform the TTR problem into a Markov decision process considering the retiming and rerouting of trains. Then, we construct the MDRL framework with the definition of state, action, transition, reward, and value function approximations with neural networks for each agent (i.e., rail train), by considering the information of different disruption events as tasks. To overcome the low training efficiency and huge memory usage in the training of MDRL, given a large number of disruptive events in the historical data, we develop a new and high-efficient training method based on a Quadratic assignment programming (QAP) model and a Frank-Wolfe-based algorithm. Our QAP model optimizes only a small number but most “representative” tasks from the historical data, while the Frank-Wolfe-based algorithm approximates the nonlinear terms in the value function of MDRL and updates the model parameters among different training tasks concurrently. Finally, based on the real-world data from the Beijing–Zhangjiakou high-speed railway systems, we evaluate the performance of our MDRL approach by benchmarking it against state-of-the-art approaches in the literature. Our computational results demonstrate that an offline-trained MDRL is able to generate near-optimal TTR solutions in real-time against different disruption scenarios, and it evidently outperforms state-of-art models regarding solution quality and computational time.