Second-order Time Delay Reservoir Computing for Nonlinear Time Series Problems

Abstract

Time Delay Reservoir (TDR) can exhibit effects of high dimensionality and short-term memory based on delay differential equations (DDEs), as well as having hardware-friendly characteristics. However, the predictive performance and memory capacity of the standard TDRs are still limited, and dependent on the hyperparameter of the oscillation function. In this paper, we first analyze these limitations and their corresponding reasons. We find that the reasons for such limitations are fused by two aspects, which are the trade-off between the strength of self-feedback and neighboring-feedback caused by neuron separation, as well as the unsuitable order setting of the nonlinear function in DDE. Therefore, we propose a new form of TDR with second-order time delay to overcome such limitations, incurring a more flexible time-multiplexing. Moreover, a parameter-free nonlinear function is introduced to substitute the classic Mackey-Glass oscillator, which alleviates the problem of parameter dependency. Our experiments show that the proposed approach achieves better predictive performance and memory capacity compared with the standard TDR. Our proposed model also outperforms six other existing approaches on both time series prediction and recognition tasks.