PXNOR: Perturbative Binary Neural Network

Abstract

Research into deep neural networks has brought about architectures and models that solve problems we once thought could not be approached by machine learning. Year after year, performance improves, to the point that it is becoming difficult to differentiate between the strengths of deep neural network models given our current data sets. However, due to their significant requirements in terms of hardware resources, all but few architectures are dependent on cloud environments. Yet, there are many use cases for neural networks in a variety of areas, many of which require consumer-grade hardware or highly resource constrained embedded devices. This paper offers a comparison of selected state-of-the-art neural network miniaturization methods, and proposes a new approach, PXNOR, that achieves a noteworthy accuracy, remarkable inference speed and significant memory savings. PXNOR seeks to fully replace traditional convolutional filters with approximate operations, while replacing all multiplications and additions with simpler, much faster versions such as XNOR and bitcounting, which are implemented at hardware level on all existing platforms.