Autonomous interval management of multi-aircraft based on multi-agent reinforcement learning considering fuel consumption

Abstract

Real-time autonomous interval management in multi-aircraft operational scenarios addresses safety, efficiency, and economic issues in air transportation. This study proposes an autonomous interval management supporter (AIMS) prototype system with high scalability potential to address these issues. The system utilizes a multi-agent deep reinforcement learning method, specifically the deep deterministic policy gradient (DDPG) algorithm, which enables interval management and fuel-saving by providing speed decisions in a continuous action space amidst uncertainty. This study innovatively incorporates aircraft performance-related parameters as observational features. These features are categorized into interval- and performance-related groups as inputs, and trained using a separate reconstructed critic network structure. Experiments are focused on the enroute descent phase to validate the performance of the proposed AIMS. Compared with real flight data based on traffic controller decisions, the AIMS demonstrated superior speed change decision-making regardless of the aircraft type or classification criteria. Simulation results suggest that incorporating aircraft performance-related states and utilizing a separate critic network training structure positively improve the success rate of decision-making and reduce fuel consumption. By utilizing aircraft performance-related states, the success rate increases by an average of 49.64%, with a corresponding average fuel consumption decrease of 4.42%. Additionally, employing a separate critic network training structure results in an average success rate increase of 16.10%, with an average fuel reduction of 1.09%. To further reduce fuel consumption and achieve a shortened interval, it is recommended to set the initial altitude of the aircraft sequence appropriately high based on flight altitude constraints.