Evolving Robust Solutions for Stochastically Varying Problems

Abstract

We demonstrate how evaluating candidate solutions in a limited number of stochastically varying conditions that vary over generations at a moderate rate is an effective method for developing high quality robust solutions. Indeed, agents evolved with this method for the ability to solve an extended version of the double-pole balancing problem, in which the initial state of the agents and the characteristics of the environment in which the agents are situated vary, show the ability to solve the problem in a wide variety of environmental circumstances and for prolonged periods of time without the need to readapt. The combinatorial explosion of possible environmental conditions does not prevent the evolution of robust solutions. Indeed, exposing evolving agents to a limited number of different environmental conditions that vary over generations is sufficient and leads to better results with respect to control experiments in which the number of experienced environmental conditions is greater. Interestingly the exposure to environmental variations promotes the evolution of convergent strategies in which the agents act so to exhibit the required functionality and so to reduce the complexity of the control problem.