An information theoretic approach to neural network based system identification

Abstract

The paper presents an approach to system identification of input/output mappings of non-linear stochastic systems in accordance to an information-theoretic criterion. At that, a parameterized description of the system under study is utilized combined with a corresponding technique of estimation of the mutual information (in the Shannon sense), leading, finally, to a problem of the finite dimensional optimization. Solving the latter is based on applying ideas of papers on using neural networks within problems of optimization of continuous functions.