A Robust Multi-Virtual-Agent Inverse Reinforcement Learning Approach With Data Aggregation for Perturbed Environments

Abstract

Learning control in environments with uncertainties and perturbations remains a challenging issue in the field of artificial intelligence. Though conventional imitation learning (IL) and inverse reinforcement learning (IRL) methods have made some progress in handling perturbations, the repeatability and resilience are somehow limited. To alleviate this issue, we propose a multi-virtual-agent IRL (MVIRL) method to produce stable policies. Specifically, we design multiple virtual agents interacting with pertinent environments. The proposed MVIRL method can recover a resilient reward function from multiple demonstration sources. This recovered reward function provides adequate information and comprehensive coverage of perturbations by considering the upper and lower bounds. Moreover, using maximum discrimination for the worst case and applying data aggregation, the proposed method requires fewer demonstrations than existing methods and improves the ability to handle uncertainties. Case studies with gravity and noise interruptions are considered to validate the effectiveness of the proposed method. The proposed MVIRL method obtains better performance than comparable IL and IRL methods in terms of average return (Avg Return) and standard deviation (SD) metrics, and it is more robust to the level of uncertainties.