A Cognitive ML Agent for Airborne Networking

Abstract

This conference note describes a Deep Reinforcement Learning architecture specifically designed to improve wireless network performance over a heterogeneous airborne wireless network consisting of multiple waveforms, antennas, platforms, link protocols, frequencies, spatial transmission, and codes. The cooperative optimization of this high dimensional space is a difficult problem which obviously has a highly correlated characterization where human network operators cannot possibly capture these correlations. Model-free Reinforcement Learning techniques represent a potential solution to our problem. Specifically, we use Deep Q-Learning Networks (DQN) to improve networking performance. We have developed a high-fidelity network simulation tool we call Tactical Airborne Network Simulator (TANS) which we use to train our neural network before deploying to the field where the asset is deployed to some mission which is hopefully somewhat similar to the scenarios used for training. By utilizing the model developed under the TANS training scenarios for the target mission scenario our learning technique gets a head start, rather than using a truly model-free approach. Our technique is codified in the ML community as “Deep Transfer Learning” [4] where terms and metrics have been examined. This paper represents an initial investigation into both the decision support agent architecture and the ML technique. Upcoming research will be described below including our vision for an expanded agent architecture as well as ideas for improved ML techniques which ultimately will result in better wireless network performance. Here we demonstrate a minor throughput performance improvement of 4% using a proof of concept agent over the use of a standard unassisted network. We improved the throughput from 309kbps to 324 kbps.