{"title":"基于深度强化学习的空中交通管理冲突解决策略","authors":"Dong Sui, Chenyu Ma, Jintao Dong","doi":"10.3846/aviation.2023.19720","DOIUrl":null,"url":null,"abstract":"With the continuous increase in flight flows, the flight conflict risk in the airspace has increased. Aiming at the problem of conflict resolution in actual operation, this paper proposes a tactical conflict resolution strategy based on Deep Reinforcement Learning. The process of the controllers resolving conflicts is modelled as the Markov Decision Process. The Deep Q Network algorithm trains the agent and obtains the resolution strategy. The agent uses the command of altitude adjustment, speed adjustment, or heading adjustment to resolve a conflict, and the design of the reward function fully considers the air traffic control regulations. Finally, simulation experiments were performed to verify the feasibility of the strategy given by the conflict resolution model, and the experimental results were statistically analyzed. The results show that the conflict resolution strategy based on Deep Reinforcement Learning closely reflected actual operations regarding flight safety and conflict resolution rules.","PeriodicalId":51910,"journal":{"name":"Aviation","volume":"80 1","pages":"0"},"PeriodicalIF":0.8000,"publicationDate":"2023-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"CONFLICT RESOLUTION STRATEGY BASED ON DEEP REINFORCEMENT LEARNING FOR AIR TRAFFIC MANAGEMENT\",\"authors\":\"Dong Sui, Chenyu Ma, Jintao Dong\",\"doi\":\"10.3846/aviation.2023.19720\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"With the continuous increase in flight flows, the flight conflict risk in the airspace has increased. Aiming at the problem of conflict resolution in actual operation, this paper proposes a tactical conflict resolution strategy based on Deep Reinforcement Learning. The process of the controllers resolving conflicts is modelled as the Markov Decision Process. The Deep Q Network algorithm trains the agent and obtains the resolution strategy. The agent uses the command of altitude adjustment, speed adjustment, or heading adjustment to resolve a conflict, and the design of the reward function fully considers the air traffic control regulations. Finally, simulation experiments were performed to verify the feasibility of the strategy given by the conflict resolution model, and the experimental results were statistically analyzed. The results show that the conflict resolution strategy based on Deep Reinforcement Learning closely reflected actual operations regarding flight safety and conflict resolution rules.\",\"PeriodicalId\":51910,\"journal\":{\"name\":\"Aviation\",\"volume\":\"80 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.8000,\"publicationDate\":\"2023-10-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Aviation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3846/aviation.2023.19720\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"ENGINEERING, AEROSPACE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Aviation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3846/aviation.2023.19720","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, AEROSPACE","Score":null,"Total":0}
CONFLICT RESOLUTION STRATEGY BASED ON DEEP REINFORCEMENT LEARNING FOR AIR TRAFFIC MANAGEMENT
With the continuous increase in flight flows, the flight conflict risk in the airspace has increased. Aiming at the problem of conflict resolution in actual operation, this paper proposes a tactical conflict resolution strategy based on Deep Reinforcement Learning. The process of the controllers resolving conflicts is modelled as the Markov Decision Process. The Deep Q Network algorithm trains the agent and obtains the resolution strategy. The agent uses the command of altitude adjustment, speed adjustment, or heading adjustment to resolve a conflict, and the design of the reward function fully considers the air traffic control regulations. Finally, simulation experiments were performed to verify the feasibility of the strategy given by the conflict resolution model, and the experimental results were statistically analyzed. The results show that the conflict resolution strategy based on Deep Reinforcement Learning closely reflected actual operations regarding flight safety and conflict resolution rules.
期刊介绍:
CONCERNING THE FOLLOWING FIELDS OF RESEARCH: ▪ Flight Physics ▪ Air Traffic Management ▪ Aerostructures ▪ Airports ▪ Propulsion ▪ Human Factors ▪ Aircraft Avionics, Systems and Equipment ▪ Air Transport Technologies and Development ▪ Flight Mechanics ▪ History of Aviation ▪ Integrated Design and Validation (method and tools) Besides, it publishes: short reports and notes, reviews, reports about conferences and workshops