Design of Logarithmic Number System for LSTM

Abstract

This paper presents a circuit architecture of the long short-term memory (LSTM) model for neural networks by applying a logarithmic number system (LNS) with an aim to reducing the hardware cost and power consumption. The proposed design exploits the piece-wise polynomial approximation technique to enhance the computing accuracy of logarithmic and anti-logarithmic converters in LNS. Results show that the computing accuracy of the developed scheme has an improvement of 81.5% only with an increase of 4.37% in the circuit area over the existing LNS. The proposed LSTM architecture is experimented and verified by the heart sound recognition classification problem. The recognition accuracy is 81.63% in the conventional binary system with IEEE754 single-precision format. The proposed LNS with the approximated activation functions shows a recognition accuracy of 74.22%, while the recognition accuracy is 71.75% by using the existing LNS. Although the accuracy is worse than that of the traditional binary system, the computing time of the presented architecture is reduced by 38.68%.