A Group Regularization Framework of Convolutional Neural Networks Based on the Impact of Lₚ Regularizers on Magnitude

Abstract

Group regularization is commonly employed in network pruning to achieve structured model compression. However, the rationale behind existing studies on group regularization predominantly hinges on the sparsity capabilities of $L_{p}$ regularizers. This singular focus may lead to erroneous interpretations. In response to these limitations, this article proposes a novel framework for evaluating the penalization efficacy of group regularization methods by analyzing the impact of $L_{p}$ regularizers on weight magnitudes and weight group magnitudes. Within this framework, we demonstrate that $L_{1,2}$ regularization, contrary to prevailing literature, indeed exhibits favorable performance in structured pruning tasks. Motivated by this insight, we introduce a hybrid group regularization approach that integrates $L_{1,2}$ regularization and group $L_{1/2}$ regularization (denoted as HGL1,2& $L_{1/2}$ ). This novel method addresses the challenge of selecting appropriate $L_{p}$ regularizers for penalizing weight groups by leveraging $L_{1,2}$ regularization for penalizing groups with magnitudes exceeding a critical threshold while employing group $L_{1/2}$ regularization for other groups. Experimental evaluations are conducted to verify the efficiency of the proposed hybrid group regularization method and the viability of the introduced framework.