Distributed Primal-Dual Proximal Method for Regularized Empirical Risk Minimization

Abstract

Most high-dimensional estimation and classification methods propose to minimize a loss function (empirical risk) that is the sum of losses associated with each observed data point. We consider the special case of binary classification problems, where the loss is a function of the inner product of the feature vectors and a weight vector. For this special class of classification tasks, the empirical risk minimization problem can be recast as a minimax optimization which has a unique saddle point when the losses are smooth functions. We propose a distributed proximal primal-dual method to solve the minimax problem. We also analyze the convergence of the proposed primal-dual method and show its convergence to the unique saddle point. To prove the convergence results, we present a novel analysis of the consensus terms that takes into account the non-Euclidean geometry of the parameter space. We also numerically verify the convergence of the proposed algorithm for the logistic regression on the Erdos-Reyni random graphs and lattices.