Parallel Trajectory Training of Recurrent Neural Network Controllers With Levenberg–Marquardt and Forward Accumulation Through Time in Closed-Loop Control Systems

Abstract

This paper introduces a novel parallel trajectory mechanism that combines Levenberg-Marquardt and Forward Accumulation Through Time algorithms to train a recurrent neural network controller in a closed-loop control system by distributing the calculation of trajectories across Central Processing Unit (CPU) cores/workers depending on the computing platforms, computing program languages, and software packages available. Without loss of generality, the recurrent neural network controller of a grid-connected converter for solar integration to a power system was selected as the benchmark test closed-loop control system. Two software packages were developed in Matlab and C++ to verify and demonstrate the efficiency of the proposed parallel training method. The training of the deep neural network controller was migrated from a single workstation to both cloud computing platforms and High-Performance Computing clusters. The training results show excellent speed-up performance, which significantly reduces the training time for a large number of trajectories with high sampling frequency, and further demonstrates the effectiveness and scalability of the proposed parallel mechanism.