Flat Neural Networks and Rapid Learning Algorithms

Abstract

In this chapter, we will introduce flat neural networks architecture. The system equations of flat neural networks can be formulated as a linear system. In this way, the performance index is a quadratic form of the weights, and the weights of the networks can be solved easily using a linear least-square method. Even though they have a linear-system-equations-like equation, the flat neural networks are also perfect for approximating non-linear functions. A fast learning algorithm is given to find an optimal weight of the flat neural networks. This formulation makes it easier to update the weights instantly for both a newly added input and a newly added node. A dynamic stepwise updating algorithm is given to update thc weights of the system instantly. Finally, we give several examples of applications of the flat neural networks, such as an infrared laser data set, a chaotic time-series, a monthly flour price data set, and a non-linear system identification problem. The simulation results are compared to existing models in which more complex architectures and more costly training are needed. The results indicate that the flat neural networks are very attractive to real-time processes. 5.1 Introduction Feedforward artificial neural networks have been a popular research subject recently. The research topics vary from the theoretical view of learning algorithms such as learning and generalization properties of the networks to a variety of applications in control, classification, biomedical, manufacturing, and business forecasting, etc. The backpropagation (BP) supervised learning algorithm is one of the most popular learning algorithms being developed for layered networks [l-21. Improving the learning speed of BP and increasing the generalization capability ofthe networks have played a center role in neural network research [3-91. Apart from multi-layer network architectures and the BP algorithm, various simplified architectures or different non-linear activation functions have been devised. Among those, so-called flat networks including functional-link neural networks and radial basis function networks have been proposed [lo-151. These flat networks remove the drawback