Short-Term Visual-IMU Fusion Memory Agent For Drone's Motion Planning

This paper provides a deep reinforcement learning motion planning algorithm that uses time series image information as input. In this paper, we use a special visual state structure. This structure is composed of two kinds of sensor data. We combine the monocular visual information compressed by the variational autoencoder (VAE) and the inertial measurement unit (IMU) data. Then we use four consecutive combinations of the sensor data as the state of the reinforcement learning model. This method greatly simplifies the neural network complexity of continuous action space deep reinforcement learning on visual control tasks. After training in a drone simulation environment, this method has been proven to enable the drone to learn to avoid obstacles of various shapes and obstacles on the side blind corner of the camera.