Gradient-Free Algorithms for Solving Stochastic Saddle Optimization Problems with the Polyak–Łojasiewicz Condition

Abstract

This paper focuses on solving a subclass of stochastic nonconvex-nonconcave black box optimization problems with a saddle point that satisfy the Polyak–Łojasiewicz (PL) condition. To solve this problem, we provide the first (to our best knowledge) gradient-free algorithm. The proposed approach is based on applying a gradient approximation (kernel approximation) to an oracle-biased stochastic gradient descent algorithm. We present theoretical estimates that guarantee its global linear rate of convergence to the desired accuracy. The theoretical results are checked on a model example by comparison with an algorithm using Gaussian approximation.