Monte Carlo method based precision analysis of deep convolution nets

Abstract

Convolution Neural Networks today provide the best results for many image detection and image recognition problems. The network accuracy increase in the past years is obtained through an increase in complexity of the structure and amount of parameters of the deep networks. Memory bandwidth and power consumption constraints are limiting the deployment of such state-of-the-art architecture in low power embedded applications. Reduced coefficient bit depth is one of the most frequently used approach to bring the deep learning neural networks into low power embedded hardware accelerators. In this paper we propose a reduced precision, fixed point implementation that can reduce bandwidth and power consumption significantly. The results show that with an 8bit representation for more than 64% of the parameters less than 0.5% accuracy is lost. As expected, the error resilience varies from layer to layer and convolution kernel to convolution kernel. To cope with this variability and understand what parameter need what type of precision we have developed a Monte Carlo simulation tool that explores the decision space.