Effective Learning Mechanism Based on Reward-Oriented Hierarchies for Sim-to-Real Adaption in Autonomous Driving Systems

Abstract

Autonomous vehicles in Intelligent Transportation Systems aim to boost the adaptability performances of complex problem-solving behaviour in the Sim-to-Real self-driving mission. However, the difficulty for Sim2Real adaption is the so-called “catastrophic forgetting” challenge, i.e., the pre-training policy exposes the flaws of the inability to retain previously skill motion when generalizing to the mixed real-world scenario, which affects learning in an inefficient way. This paper could deal with the above challenge by taking advantage of reconfigurable Sim2Real policies from simpler, previously learned sub-tasks, which are superior to those of pre-defined artificial systems. Specifically, a novel reward-oriented hierarchical learning framework based on hierarchical cognitive mechanisms is proposed dedicated to Sim2Real autonomous driving. Such a learning mechanism breaks down the behavior-aware experience into two distinguished types concerning environmental rewards: basic task-agnostic background and dynamic object-specific foreground. It further reveals the intrinsic association between previously learned knowledge and multiple changing events, by utilizing goal-conditioned key skill motion tailored for specific sub-task rewards. Moreover, the reconfigurable Sim2Real rehearsal is developed to boost the efficiency of high-level policies’ generalization ability according to the reuse of the configurable skill motion via mirrored composition. Extensive validation on both simulated and real-world Sim2Real testbench of challenging autonomous driving scenarios outperforms, demonstrating the superiority of the proposed learning mechanism in improving task efficiency and handling stochasticity throughout learning.