Evolving Complementary Sparsity Patterns for Hardware-Friendly Inference of Sparse DNNs

Sparse deep learning models are known to be more accurate than their dense counterparts for equal parameter and computational budgets. Unstructured model pruning can deliver dramatic compression rates, yet the consequent irregular sparsity patterns lead to severe computational challenges for modern computational hardware. Our work introduces a set of complementary sparsity patterns to construct both highly expressive and inherently regular sparse neural network layers. We propose a novel training approach to evolve inherently regular sparsity configurations and transform the expressive power of the proposed layers into a competitive classification accuracy even under extreme sparsity constraints. The structure of the introduced sparsity patterns engenders optimal compression of the layer parameters into a dense representation. Moreover, the constructed layers can be processed in the compressed format with full-hardware utilization in minimally modified non-sparse computational hardware. The experimental results demonstrate superior compression rates and remarkable performance improvements in sparse neural network inference in systolic arrays.