DRL-AdCAR: Adaptive Coding-Aware Routing With Maximum Coding Opportunities and High-Quality via Deep Reinforcement Learning in FANET

Abstract

Research on flying ad-hoc networks (FANETs) has become important with the development of unmanned aerial vehicle (UAV) systems. Routing design in FANET is a major challenge due to its inherent characteristics, including dynamic network topology and network self-organization. Coding-aware routing based on network coding improves the performance of the network by selecting paths with more coding opportunities. However, current methods are mostly used in networks with relatively fixed topologies, which are difficult to adapt to the FANET environment. To address these issues, we propose an adaptive coding-aware routing algorithm via deep reinforcement learning (DRL-AdCAR). First, we transform the routing problem into a Markov decision model. Then, coding opportunities, coding gains, and link quality are considered simultaneously in the reward function to avoid the drawbacks of coding-aware routing algorithms that simply aim to increase coding opportunities while greatly affecting other aspects of network performance. In addition, we present an improvement of the deep deterministic policy gradient (DDPG) algorithm for FANET, combining the gated recurrent unit (GRU) and long short-term memory networks (LSTM) algorithms to replace the traditional neural network structure, which ensures prediction accuracy and improves the training efficiency. The experimental results showed that DRL-AdCAR can adaptively select the transmission paths with the most suitable coding opportunities according to environmental changes in FANET, improving coding performance and enhancing the network throughput and packet delivery rate.