A cross-platform deep reinforcement learning model for autonomous navigation without global information in different scenes

This paper employs a deep reinforcement learning algorithm named Twin Delayed Deep Deterministic algorithm into autonomous navigation in intelligent transportation systems. It trains a fully connected neural network model in a simulation environment, which outputs the expected linear and angular velocity of the vehicle based on real-time data measured by embedded sensors. Through continuous epochs of training, the model gradually navigates the vehicle to reach a provided destination by making rational motion decisions at each discrete time instant without knowing global environment information. Especially, to improve the model’s generalization ability across various scenes, an input preprocessing function is proposed to eliminate the singularity and uniformity of raw input data. A large number of simulation tests are carried out, where the proportion that the vehicle moves from a start position to a destination without collision within a specified limited time exceeds 90%. The remaining failures are mainly due to the vehicle’s inability to approach the destination immediately adjacent to obstacles for its safety. Furthermore, traditional mapless navigation algorithms suffer from locally optimal solutions in the face of U-shaped obstacles. This paper introduces a virtual obstacle mechanism designed to prevent the vehicle from entering the U-shaped region, effectively addressing the aforementioned issue. Finally, the model trained from the simulation environment can be directly loaded onto a physical vehicle without considering the different processor architectures. Large quantities of experiments show that the model improves the autonomous navigation capability of vehicles when global environment information cannot be obtained by the system, which optimizes the functions of the navigation module in intelligent transportation systems.