Assembly of Echo State Networks Driven by Segregated Low Dimensional Signals

Abstract

An echo state network (ESN), consisting of an input layer, reservoir, and output layer, provides a higher learning-efficient approach than other recurrent neural networks (RNNs). In the design of ESNs, a sufficiently large number of reservoir neurons is required compared to the dimension of the input signal. Thus, the number of neurons must be increased for high-dimensional input to achieve good performance. However, an increase in the number of neurons increases the computational load. To solve this problem, we propose an assembly ESN (AESN) architecture comprising a feature extraction part that uses multiple sub-ESNs with segregated components of high-dimensional input and a feature integration part. To validate the effectiveness of the proposed AESN, we investigated and compared the conventional ESN with the AESN under high-dimensional input. The results show that the AESN is possibly superior to the conventional ESN in accuracy, memory performance, and computational load. We believe that the AESN also has a correct integration function. Therefore, the proposed method is expected to solve high-dimensional problems with improved accuracy.