ZIZO: A Novel Zoom-In–Zoom-Out Search Algorithm for the Global Parameters of Echo-State Networks

Abstract

Echo-state networks (ESNs) are a distinct architecture for recurrent neural networks (RNNs). The great advantage of ESN is that they offer an easy way to train the RNN. To make full use of ESN, one needs to first identify their global (hyper) parameters. These are input scaling, leaking rate (for leaky ESN), spectral radius, and the size of the ESN. The most recommended way to get their optimal (or suboptimal) values is by trial-and-error. However, in practice, this method has a very low efficiency. In order to tackle this problem, we propose a novel “zoom-in-zoom-out” (ZIZO) algorithm for generating the global parameters automatically. The proposed technique consists of two major parts. First, we generate random ranges for the parameters of ESNs. Then, based on bootstrap-sampling, we search the optimal solution within the fixed specific ranges. To evaluate the proposed method, we use two different data sets, which are collected from the literature. The obtained results demonstrate the efficiency and accuracy of ZIZO.