Evolutionary Pruning of Deep Convolutional Networks by a Memetic GA with Sped-Up Local Optimization and GLCM Energy Z-Score

Abstract

This paper introduces a novel method of selecting the most significant filters in deep neural networks. We performed model simplification via pruning with Genetic Algorithm (GA) for trained deep networks. Pure GA has a weakness of local tuning and slow convergence, so it is not easy to produce good results for problems with large problem space such as ours. We present new ideas that overcome some of GA's weaknesses. These include efficient local optimization, as well as reducing the time of evaluation which occupies most of the running time. Additional time was saved by restricting the filters to preserve using the GLCM (Gray-Level Co-occurrence Matrix) to determine the usefulness of the filters. Ultimately, the saved time was used to perform more iterations, providing the opportunity to further optimize the network. The experimental result showed more than 95% of reduction in forward convolution computation with negligible performance degradation.