Robust sparse orthogonal basis clustering for unsupervised feature selection

Abstract

Unsupervised Feature Selection (UFS), which identifies the optimal-related feature subset from the original feature set to lower the dimensionality of data without label information, has had a high profile in recent years. Given the absence of label information, the existing UFS approaches usually utilize graph and manifold learning techniques to retain the intrinsic structure of the data. The inclusion of irrelevant and redundant features and noise, would inevitably lower the quality of the structure. For this purpose, in this paper, we come up with Robust Sparse Orthogonal Basis Clustering (RSOBC), a novel method for UFS that integrates feature selection process with clustering task into a unified framework. Instead of explicitly utilizing the pre-computed local information, such a strategy focuses on exploring the inherent clustering structures of data. RSOBC leverages the log-based function as the loss to lessen the effect of noise and outliers, thereby enhancing its robustness. To select the more useful and discriminative features, the ${\ell }_{2,1}$ norm is employed as the sparse regularization to encourage sparsity of the projection matrix. Meanwhile, we adopt the low redundancy regularization to make the weights of the correlated features small. In this way, the correlated features cannot be selected simultaneously. Consequently, the projection matrix, centroid matrix and cluster label matrix are learned simultaneously, such that the intrinsic structure is constructed in a more accurate way. The resulting optimization can be readily tackled by multi-block Alternating Direction Method of Multipliers (ADMM) based algorithm. Comprehensive experiments have been carried out on nine diverse real-world datasets. The results demonstrate that RSOBC surpasses many state-of-the-art UFS approaches, which indicates its effectiveness and superiority.