Dual Averaging for Distributed Unbalanced Optimization With Delayed Information

Abstract

In this paper, we study a category of distributed constrained optimization problems where each agent has access to local information, communicates with its neighbors, and cooperatively minimizes the aggregated cost functions over time-varying unbalanced graphs. To address the considered problems, we propose a distributed dual averaging algorithm based on a row-stochastic weighted matrix (DDAR), which improves the robustness of network topology compared to conventional push-sum algorithms. Moreover, we develop a modified version of DDAR with delayed information (DDARD), which considers the delays of both network communication and gradient calculation, enhancing the algorithm's flexibility in communication and iteration. Our analysis demonstrates that the DDAR and DDARD achieve the optimal value at rates of ${\mathcal {O}}(\frac{N}{(1-\lambda)\sqrt{T}})$ and ${\mathcal {O}}(\frac{{\tilde{\tau }}_{}^{2}N}{(1-{\tilde{\lambda }})\sqrt{T}})$, respectively. Finally, the theoretical results are confirmed by simulation on a logistic regression problem.