An Efficient Hardware Implementation of Activation Functions Using Stochastic Computing for Deep Neural Networks

Abstract

In this paper, we present a new approximation method for non-linear activation functions including tanh and sigmoid functions using stochastic computing (SC) logic based on the piecewise-linear approximation (PWL) for the full range of [-1, 1]. SC implementations with PWL approximation expansions for non-linear functions are based on a 90nm CMOS process. The implementation results shown that the proposed SC circuits can provide better performance compared with the previous methods such as the well-known Maclaurin expansions based, Bernstein polynomial based and finite-state-machine (FSM) based implementations. The implementation results are also presented and discussed.