Quasi-Stochastic Approximation: Design Principles With Applications to Extremum Seeking Control

How can you optimize a function $\Gamma{:}{\mathfrak{R}}^{d}\rightarrow{\mathfrak{R}}$ based on evaluations of this function without access to its gradient? Kiefer and Wolfowitz proposed a solution in the early 1950s based on stochastic approximation (SA) [16] , and in the 1920s, an engineer for the French railway system proposed an entirely deterministic approach that is now known as extremum seeking control ( ESC ) [27] , [48] . Once you understand the ESC architecture, you will find that the ideas are very similar. A fundamental difference is that random noise is replaced with sinusoids for exploration.