Hardware Realization of Sigmoid and Hyperbolic Tangent Activation Functions

Abstract

Artificial neural networks have gradually become omnipresent to the extent that they are recognised as the explicit solution to innumerable practical applications across various domains. This work aims to propose a novel hardware architecture for implementing the activation functions recurrently employed in artificial neural networks. The approach involves the development of a new hardware for the sigmoid and hyperbolic tangent activation functions based on the optimised polynomial approximations, which comprises of the critical half of realising neural Networks in general and recurrent neural networks in particular.