Heterogeneous Imitation Learning from Demonstrators of Varying Physiology and Skill

Abstract

Imitation learning enables a learner to improve its abilities by observing others. Most robotic imitation learning systems only learn from demonstrators that are homogeneous physiologically (i.e. the same size and mode of locomotion) and in terms of skill level. To successfully learn from physically heterogeneous robots that may also vary in ability, the imitator must be able to abstract behaviours it observes and approximate them with its own actions, which may be very different than those of the demonstrator. This paper describes an approach to imitation learning from heterogeneous demonstrators, using global vision for observations. It supports learning from physiologically different demonstrators (wheeled and legged, of various sizes), and self-adapts to demonstrators with varying levels of skill. The latter allows a bias toward demonstrators that are successful in the domain, but also allows different parts of a task to be learned from different individuals (that is, worthwhile parts of a task can still be learned from a poorly-performing demonstrator). We assume the imitator has no initial knowledge of the observable effects of its own actions, and train a set of Hidden Markov Models to map observations to actions and create an understanding of the imitator's own abilities. We then use a combination of tracking sequences of primitives and predicting future primitives from existing combinations using forward models to learn abstract behaviours from the demonstrations of others. This approach is evaluated using a group of heterogeneous robots that have been previously used in RoboCup soccer competitions.