Distributed No-Regret Learning in Aggregative Games With Residual Bandit Feedback

Abstract

This article investigates distributed no-regret learning in repeated aggregative games with bandit feedback. The players lack an explicit model of the game and can only learn their actions based on the sole available feedback of payoff values. In addition, they cannot directly access the aggregate term that contains global information, while each player shares information with its neighbors without revealing its own strategy. We present a novel no-regret learning algorithm named distributed online gradient descent with residual bandit. In the algorithm, each player maintains a local estimate of the aggregate and adaptively adjusts its next action through the residual bandit mechanism and the online gradient descent method. We first provide regret analysis for aggregative games where the player-specific problem is convex, showing crucial associations between the regret bound, network connectivity, and game structure. Then, we prove that when the game is also strictly monotone, the action sequence generated by the algorithm converges to the Nash equilibrium almost surely. Finally, we demonstrate the algorithm performance through numerical simulations on the Cournot game.