Improvement of Dynamic Hybrid Collaborative Filtering Based on Spark

Abstract

Iterative computation due to the advantage of memory computing framework in Spark big data platform, so This paper applies ALS model recommendation algorithm on Spark platform and improves its calculation method. Considering more practical factors to get more accurate result sets, we first use C-Means clustering to classify data preprocessing, so as to reduce the calculation of redundant data and the sparsity of matrix. Secondly, the cosine similarity and Pearson similarity are applied to improve the user similarity calculation. Finally, a mixed recommendation function is constructed. On the Spark distributed large data platform, this method trains and compares the results offline and real-time through MovieLens data set, which shows that it reduces the computing time, improves the efficiency and accuracy of the algorithm.