Global critic and local actor for campaign-tactic combat management in the joint operation simulation software

Abstract

Users of the simulation software not only need to model the capability of each unit, but also need to create a decision maker for the units in the simulation software, to control ships, aircrafts and ground units to cooperates to achieve one goal. In this paper a new approach is constructed to create the decision maker. We use reinforcement learning based on global critic and local actor. The invention constructs an air isomorphic formation command method based on multiagent PPO algorithm. The evaluation network uses global information, so that the algorithm has the ability to evaluate global information and guide the agent to select actions that are beneficial to the global environment state. The input of the action network is local information, so that the agent can focus on local information.