Towards effective processing of large text collections

Abstract

In the article we describe the approach to parallel implementation of elementary operations for textual data categorization. In the experiments we evaluate parallel computations of similarity matrices and k-means algorithm. The test datasets have been prepared as graphs created from Wikipedia articles related with links. When we create the clustering data packages, we compute pairs of eigenvectors and eigenvalues for visualizations of the datasets. We describe the method used for evaluation of the clustering quality. Finally we discuss achieved results, point some improvements and perspectives for future development.