An Improved GAIL Based on Object Detection, GRU, and Attention

Imitation Learning (IL) learns expert behavior without any reinforcement signal. Thus, it is seen as a potential alternative to Reinforcement Learning (RL) in tasks where it is not easy to design reward functions. However, most models based on IL methods cannot work well when the demonstration is high dimension, and the tasks are complex. We set one realistic-like UAV race simulation environment on AirSim Drone Racing Lab (ADRL) to study the two problems. We propose a new model improves on Generative Adversarial Imitation Learning (GAIL). An object detection network trained by the expert dataset allows the model to use high-dimensional visual inputs while alleviating the data inefficiencies of GAIL. Benefit from the recurrent structure and attention mechanism, the model can control the drone cross the gates and complete the race as if it were an expert. Compared to the primitive GAIL structure, our improved structure showed a 70.6% improvement in average successful crossing over 2000 flight training sessions. The average missed crossing decreased by 18.8% and the average collision decreased by 14.1%.