Research on Evasion Policy of UCAV Against Infrared Air-to-Air Missile Based on Soft Actor Critic Algorithm

Aiming at the policy problem of unmanned combat aerial vehicle (UCAV) evading infrared air-to-air missiles in close air combat, based on the establishment of an infrared offensive and defensive confrontation simulation system, the application of soft actor critic algorithm is studied to train the agent to learn the escape maneuver policy and decoy launching policy of UCAV to evade missiles. The three-dimensional coordinates of the missile in the UCAV body coordinate system and the number of remaining decoys are taken as the input states. The joystick, throttle stick control stroke and the decoy launching pulse are taken as the output actions. The dense reward composed of relative situational parameters and flight parameters and the sparse reward constituted by the result of the decoy interference and the result of the engagement are designed. The soft actor critic (SAC) algorithm is improved to adapt to the action space of mixed continuous action and discrete action, and finally the end-to-end UCAV escape maneuver policy and decoy launching policy from state input to control output is obtained. Through simulation verification, the escape rates of the UCAV with and without the decoys are compared, and the results show that the escape rate with escape maneuver policy realized by the agent can reach 59.0%, and the escape rate combined with the decoy launching policy will increase by 6.7%, finally the UCAV escape rate can reach 65.7%.