Multi-level sparse network lasso: Locally sparse learning with flexible sample clusters

Abstract

Traditional learning usually assumes that all samples share the same global model, which fails to preserve critical local information for heterogeneous data. It can be tackled by detecting sample clusters and learning sample-specific models but is limited to sample-level clustering and sample-specific feature selection. In this paper, we propose multi-level sparse network lasso (MSN Lasso) for flexible local learning. It multiplicatively decomposes model parameters into two components: One component is for coarse-grained group-level, and another is for fine-grained entry-level. At the clustering stage, MSN Lasso simultaneously groups samples (group-level) and clusters specific features across samples (entry-level). At the feature selection stage, it enables both across-sample (group-level) and sample-specific (entry-level) feature selection. Theoretical analysis reveals a potential equivalence to a jointly regularized local model, which informs the development of an efficient algorithm. A divide-and-conquer optimization strategy is further introduced to enhance the algorithm’s efficiency. Extensive experiments across diverse datasets demonstrate that MSN Lasso outperforms existing methods and exhibits greater flexibility.