Perceptrons with polynomial post-processing

Abstract

Introduces tensor-product neural networks, composed of a layer of univariate neurons followed by a net of polynomial post-processing. We look at the general approximation problem by these networks observing in particular their relationship to the Stone-Weierstrass theorem for uniform function algebras. The implementation of the post-processing as a two-layer network with logarithmic and exponential neurons leads to potentially important 'generalised' product networks which, however, require a complex approximation theory of the Mu/spl uml/ntz-Szasz-Ehrenpreis type. A backpropagation algorithm for product networks is presented and used in three computational experiments. In particular, approximation by a sigmoid product network is compared to that of a single-layer radial basis network and a multiple-layer sigmoid network.