A novel approach combining recurrent neural network and support vector machines for time series classification

Abstract

Echo state network (ESN) is a relatively recent type of recurrent neural network that has proved to achieve state-of-the-art performance in a variety of machine-learning tasks. This robust performance that incorporates the simplicity of ESN implementation has led to wide adoption in the machine-learning community. ESN's simplicity stems from the weights of the recurrent nodes being assigned randomly, known as the reservoir, and weights are only learnt in the output layer using a linear read-out function. In this paper, we present a novel approach that combines ESN with support vector machines (SVMs) for time series classification by replacing the linear read-out function in the output layer with SVMs with the radial basis function kernel. The proposed model has been evaluated with an Arabic digits speech recognition task. The well-known Spoken Arabic Digits Dataset, which contains 8800 instances of Arabic digits 0-9 spoken by 88 different speakers (44 males and 44 females) was used to develop and validate the suggested approach. The result of our system can be compared to the state-of-the-art models introduced by Hammami et al. (2011) and P. R. Cavalin et al. (2012) , which are the best reported results found in the literature that used the same dataset. The result shows that ESN and ESNSVMs can both provide superior performance at a 96.91% and 97.45% recognition accuracy, respectively, compared with 95.99% and 94.04% for other models. The result also shows that when using a smaller reservoir size significant differences exist in the performance of ESN and ESNSVMs, as the latter approach achieves higher accuracy by more than 15% in extreme cases.