Research on Intelligent Maneuvering Decision in Close Air Combat Based on Deep Q Network

For the Unmanned Combat Aerial Vehicle(UCAV)maneuvering decision in close air combat, the design of reinforcement learning(RL) reward function and the selection of hyperparameters are studied based on the deep Q network algorithm. Considering the angle, range, altitude, and speed factors, an auxiliary reward function is proposed to solve the sparse reward problem of RL. Meanwhile, aiming at the issue of hyperparameter selection in RL, the influence of learning rate, the number of network nodes, and layers on the decision-making system is explored, and a suitable range of parameters is given, which provides a reference for the subsequent research on parameter selection. In addition, the simulation results show that the trained agent can obtain the optimal maneuver strategy in different air combat situations, but it is sensitive to RL hyperparameters.