A Momentum-incorporated Fast Parallelized Stochastic Gradient Descent for Latent Factor Model in Shared Memory Systems

Abstract

Latent factor (LF) model is an effective method for extracting useful knowledge from high-dimensional and sparse (HiDS) data generated by various industrial applications. Parallelized stochastic gradient descent (SGD) is widely used in building a parallelized LF model for handling large-scale HiDS data, but parallelized SGD suffers from slow convergence and considerable time cost. To address this issue, this study incorporates the principle of momentum into parallelized SGD, where momentum decay coefficient and learning rate are adjusted dynamically, and proposes a momentum-incorporated fast parallelized SGD (MFSGD) method to discover latent patterns from large-scale HiDS data. The experiments on two datasets show that the proposed MFSGD method outperforms state-of-the-art parallel SGD methods in terms of computational efficiency.