A Distributed Adaptive Second-Order Latent Factor Analysis Model

Abstract

Dear Editor, This letter presents a distributed adaptive second-order latent factor (DAS) model for addressing the issue of high-dimensional and incomplete data representation. Compared with first-order optimizers, a second-order optimizer has stronger ability in approaching a better solution when dealing with the non-convex optimization problems, thus obtaining better performance in extracting the latent factors (LFs) well representing the known information from high-dimensional and incomplete data. However, a traditional second-order optimizer are inefficient in exploiting the curvature information of an LF model due to its large number of parameters. In order to reduce the computational overhead, an inexact second-order method relying on the Hessian-free optimization is preferred. However, this method requires careful coordination of its components, which is time-consuming and impractical for real applications. To address the above issues, the DAS model leverages the curvature information with a Hessian-vector-incorporated inexact second-order optimizer and embeds it into a distributed, multi-phase, and multi-elitist learning particle swarm optimization (DM2PSO) framework for efficient hyper-parameters adaptation and performance gain. Experimental results demonstrate that the DAS model outperforms several state-of-the-art models in estimating missing data on several high-dimensional and incomplete datasets from real-world applications.