Novelty search in neuroevolution for end effector positioning

Abstract

The article considers the use of the neuroevolution algorithm for neural network policies search when creating a controller for a robotic arm, in particular for the subtask of positioning the end effector. Neuro-evolution is a family of machine learning methods that use evolutionary algorithms by imitating the process of natural selection. This approach has been found to be particularly effective for the positioning task, where the final position can be achieved in many optimal ways and therefore requires reinforcement learning. It is noted that the final result of neuroevolution is an optimal network topology, which makes the model more resource-efficient and easier to analyze. The paper considers the process of neural network policy search for controlling a two-dimensional robot with two links. According to the results of the experiments, an increase in the efficiency of the best solution found using novelty search for the NEAT algorithm is noted compared to the NEAT algorithm without novelty search. It was established that the proposed approach allows to obtain an effective neural network policy, which has a minimal configuration, which will allow to increase the speed of the controller, that is critical for the operation of a real system. Thus, the use of novelty search as a method of optimizing the neuroevolutionary process to solve the positioning problem allows to increase the efficiency of the learning process and obtain the optimal network topology.