Mathematical Essence and Structures of Feedback Neural Networks and Weight Matrix Design

Abstract

This chapter focuses on mathematical essence and structures of neural networks and fuzzy neural networks, especially on discrete feedback neural networks. We begin with review of Hopfield networks and discuss the mathematical essence and the structures of discrete feedback neural networks. First, we discuss a general criterion on the stability of networks, and we show that the energy function commonly used can be regarded as a special case of the criterion. Second, we show that the stable points of a network can be converted as the fixed points of some function, and the weight matrix of the feedback neural networks can be solved from a group of systems of linear equations. Last, we point out the mathematical base of the outer-product learning method and give several examples of designing weight matrices based on multifactorial functions. In previous chapters, we have discussed in detail the mathematical essence and structures of feedforward neural networks. Here, we study the mathematical essence and structures of feedback neural networks, namely, the Hopfield networks [l]. illustrates a single-layer Hopfield net with n neurons, where are outer input variables, which usually are treated as " the first impetus " , then they are removed and the network will continue to evolve itself. connection weights, wij = wji and wii=O. The activation functions of the neurons are denoted by cpi, where the threshold values are 8i.