Evolution Strategies with Seed Mirroring and End Tournament

Abstract

This paper introduces two enhancements that apply to evolution strategies such as Augmented Random Search (ARS). These improvements target generalizable tasks with widely varying initial conditions, such as legged robot locomotion where the robot starts off in a random joint configuration. The first innovation builds upon the mirrored sampling feature of ARS. It mitigates the detrimental effect of unexplained variance on training stability by forcing the simulator to use the same random seed for both mirrored pairs. The second innovation is a multi-phase end tournament procedure performed right after the ARS method is complete. This tournament helps to ensure that the final product of training, a single model selected from the population, performs well over a wide range of random initial conditions. Improved results are demonstrated using MuJoCo simulations of legged robots.