Reinforcement Learning Environment for Tactical Networks

Abstract

Providing situational awareness is a crucial requirement and a challenging task in the tactical domain. Tactical networks can be characterized as Disconnected, Intermittent and Limited (DIL) networks. The use of cross-layer approaches in DIL networks can help to better utilize the tactical communications resources and thus improve the overall situational awareness perceived by the user. The specification of suitable cross-layer strategies (heuristics) which describe the rules for optimizing the applications remains a challenging task. We introduce an architectural concept which proposes the use of decentralized, machine learning based reinforcement agents to improve the use of network resources in DIL networks. This approach shall lead to more sophisticated strategies which are learned autonomously by the agents. As basis for the training of such reinforcement learning (RL) agents, an architecture for a learning environment is introduced. Since for the training of these agents a large number of scenarios is needed, an additional tactical model is defined. The purpose of the tactical model is to generate scenarios with dynamically changing network conditions and dynamic information exchanges between the applications and thus build the basis for training the RL agents. The tactical model itself is also based on RL agents, which simulate military units in a war gaming environment.