18. Stochastic Approximation, Momentum, and Nash Play

Abstract

Main objects here are normal-form games, featuring uncertainty and noncooperative players who entertain local visions, form local approximations, and hesitate in making large, swift adjustments. For the purpose of reaching Nash equilibrium, or learning such play, we advocate and illustrate an algorithm that combines stochastic gradient projection with the heavyball method. What emerges is a coupled, constrained, second-order stochastic process. Some friction feeds into and stabilizes myopic approximations. Convergence to Nash play obtains under seemingly weak and natural conditions, an important one being that accumulated marginal payo¤s remains bounded above.