A Decentralized Primal-Dual Method With Quasi-Newton Tracking

This paper considers the decentralized optimization problem of minimizing a finite sum of strongly convex and twice continuously differentiable functions over a fixed-connected undirected network. A fully decentralized primal-dual method (DPDM) and its generalization (GDPDM), which allows for multiple primal steps per iteration, are proposed. In our methods, both primal and dual updates use second-order information obtained by quasi-Newton techniques which only involve matrix-vector multiplication. Specifically, the primal update applies a Jacobi relaxation step using the BFGS approximation for both computation and communication efficiency. The dual update employs a new second-order correction step. We show that the decentralized local primal updating direction on each node asymptotically approaches the centralized quasi-Newton direction. Under proper choice of parameters, GDPDM including DPDM has global linear convergence for solving strongly convex decentralized optimization problems. Our numerical results show both GDPDM and DPDM are very efficient compared with other state-of-the-art methods for solving decentralized optimization.