Accelerated SVD-based initialization for nonnegative matrix factorization

Abstract

Nonnegative matrix factorization (NMF) is a popular dimensionality reduction technique. NMF is typically cast as a non-convex optimization problem solved via standard iterative schemes, such as coordinate descent methods. Hence the choice of the initialization for the variables is crucial as it will influence the factorization quality and the convergence speed. Different strategies have been proposed in the literature, the most popular ones rely on singular value decomposition (SVD). In particular, Atif et al. (Pattern Recognit Lett 122:53–59, 2019) have introduced a very efficient SVD-based initialization, namely NNSVD-LRC, that overcomes the drawbacks of previous methods, namely, it guarantees that (i) the error decreases as the factorization rank increases, (ii) the initial factors are sparse, and (iii) the computational cost is low. In this paper, we improve upon NNSVD-LRC by using the low-rank structure of the residual matrix; this allows us to obtain NMF initializations with similar quality to NNSVD-LRC (in terms of error and sparsity) while reducing the computational load. We evaluate our proposed solution over other NMF initializations on several real dense and sparse datasets.