Computational cost reduction of long short-term memory based on simultaneous compression of input and hidden state

Abstract

Long short-term memory (LSTM) has been successfully applied to acoustic modeling for automatic speech recognition (ASR). However, because of its complicated structure, LSTM requires high computational cost especially when the number of dimensions of memory cell is sufficiently high to get good ASR performance. In this paper, we present a novel technique to reduce computational cost of LSTM in which the input and the previous hidden state vectors are simultaneously compressed with a linear projection layer. From experimental results, it is shown that the proposed technique outperforms a standard LSTM and an LSTM with a recurrent projection layer. It is also shown that in the proposed technique ASR performance is improved by increasing the number of dimensions of memory cell when the sizes of models are comparable.