Implementing empirical modelling techniques with recurrent neural networks

A modelling technique, using recurrent networks, based on the NARMAX framework (Nonlinear Autoregressive Moving Average with Exogenous Inputs), is developed. Some properties of the technique are demonstrated by means of a mathematical example. In the NARMAX model, the term N indicates that the model is based on nonlinear equations, AR indicates that previous observations (y) are used, MA indicates that previous errors (e) are used and X indicates that exogenous inputs (u) are used. Often, the number of delay lines on each input type is mentioned together with the type of model. The proposed solution to the delay length problem is to use a fully recurrent neural network with the RTRL algorithm (R.J. Williams and D. Zipser, 1989) as learning scheme.