Analysis of Multilayer Perceptron with Rectifier Linear Unit Activation Function

Abstract

The implementation of analog neural network and online analog learning circuits based on memristive crossbar has been intensively explored in the recent years. The design of various activation functions is important for neuromorphic circuits and systems, especially deep leaning neural networks. There are several implementations of sigmoid and tangent activation function, while the analog hardware implementation of the neural networks with linear activation functions is an open problem. Therefore, this paper introduces a multilayer perceptron design with linear activation function using TSMC $130 \mu m$CMOS technology. In this paper, the performance of the proposed linear activation function is illustrated. In addition, the temperature variation and noise analysis are shown.