A novel neural-network gradient optimization algorithm based on reinforcement learning

Abstract

Searching appropriate step size and hyperparameter is the key to getting a robust convergence for gradient descent optimization algorithm. This study comes up with a novel gradient descent strategy based on reinforce learning, in which the gradient information of each time step is expressed as the state information of markov decision process in iterative optimization of neural network. We design a variable-view distance planner with a markov decision process as its recursive core for neural-network gradient descent. It combines the advantages of model-free learning and model-based learning, and fully utilizes the state transition information of the optimized neural-network objective function at each step. Experimental results show that the proposed method not only retains the merits of the model-free asymptotic optimal strategy but also enhances the utilization rate of samples compared with manually designed optimization algorithms.