The Generalization Ability of Artificial Neural Networks in Forecasting TCP/IP Traffic Trends: How Much Does the Size of Learning Rate Matter?

Abstract

Artificial Neural Networks (ANNs) have attracted increasing attention from researchers in many fields. They have proved to be one of the most powerful tools in the domain of forecasting and analysis of various time series. The ability to model almost any kind of function regardless of its degree of nonlinearity, positions ANNs as good candidates for predicting and modelling self-similar time series such as TCP/IP traffic. Inspite of this, one of the most difficult and least understood tasks in the design of ANN models is the selection of the most appropriate size of the learning rate. Although some guidance in the form of heuristics is available for the choice of this parameter, none have been universally accepted. In this paper we empirically investigate various sizes of learning rates with the aim of determining the optimum learning rate size for generalization ability of an ANN trained on forecasting TCP/IP network traffic trends. MATLAB Version 7.4.0.287’s Neural Network toolbox version 5.0.2 (R2007a) was used for our experiments. The results are found to be promising in terms of ease of design and use of ANNs. We found from the experiments that, depending on the difficulty of the problem at hand, it is advisable to set the learning rate to 0.1 for the standard Backpropagation algorithm and to either 0.1 or 0.2 if used in conjunction with the momentum term of 0.5 or 0.6. We advise minimal use of the momentum term as it greatly interferes with the training process of ANNs. Although the information obtained from the tests carried out in this paper is specific to the problem considered, it provides users of Back-propagation networks with a valuable guide on the behaviour of ANNs under a wide range of operating conditions. It is important to note that the guidelines accrued from this paper are of an assistive and not necessarily restrictive nature to potential ANN modellers.