Accelerated SGD for Tensor Decomposition of Sparse Count Data

Abstract

The rapid growth in the collection of high-dimensional data has led to the emergence of tensor decomposition, a powerful analysis method for the exploration of multidimensional data. Since tensor decomposition can extract hidden structures and capture underlying relationships between variables, it has been used successfully across a broad range of applications. However, tensor decomposition is a computationally expensive task, and existing methods developed to decompose large sparse tensors of count data are not efficient enough when being performed with limited computing resources. Therefore, we propose AS-CP, a novel algorithm to accelerate convergence of the stochastic gradient descent based CANDECOMP/PARAFAC (CP) decomposition model through an extrapolation method. The proposed framework can be easily parallelized in an asynchronous way. Our empirical results on three real-world datasets demonstrate that AS-CP decreases the total computation time and scales readily to large datasets without necessitating a high-performance computing platform or environment. The advantage of AS-CP over several state-of-the-art methods is also shown through a machine learning task as the computed factors by AS-CP can help identify better clinical characteristics from EHR data.