Echo-state networks (ESNs) are a distinct architecture for recurrent neural networks (RNN). The great advantage of ESN is that they offer an easy way to train the RNN. To make full use of ESN, one needs to first identify their global (hyper) parameters. These are input scaling, leaking rate (for leaky ESN), spectral radius and the size of the ESN. The most recommended way to get their optimal (or sub-optimal) values is by trial-and-error. However, in practice, this method has a very low efficiency. In order to tackle this problem, we propose a novel "Zoom-In-Zoom-Out" (ZIZO) algorithm for generating the global parameters automatically. The proposed technique consists of two major parts. First, we generate random ranges for the parameters of ESNs. Then, based on bootstrap sampling, we search the optimal solution within the fixed specific ranges. To evaluate the proposed method, we use two different data sets which are collected from literature. The obtained results demonstrate the efficiency and accuracy of ZIZO.
To appear in CJCE'17, the Canadian Journal of Electrical and Computer Engineering, 2017, IEEE.
*This work was partially supported by the Artemis EMC2 Award, the
NSF-Frontiers Cyber-Physical Heart Award, FWF-NFN RiSE Award,
FWF-DC LMCS Award, FFG Harmonia Award, FFG Em2Apps Award, and the
TUW CPPS-DK Award.