Determine the Architecture of ANNs by Using the Peak Search Algorithm and Delta Values

Abstract

The solution obtained by an Artificial Neural Network does not guarantee that it always yields with the simplest neural network architecture for particular problem. This causes computational complexity of training, deployment, and usage of the trained of an artificial neural network. It has observed that the hidden layer architecture of an artificial neural network significantly influences on its solution. However, still modeling of the hidden layer architecture of an artificial neural network remains as a research challenge. This paper presents a theoretically-based approach to prune hidden layers of trained artificial neural networks, ensuring better or the same performance of a simpler network as compared with the original network and then discusses how to extend the proposed method to deep learning nets. The method was inspired by the facts of neuroplasticity. It achieves the solution by two phases. First, the number of hidden layers is determined by using a peak search algorithm and then newly discovered simpler network with lesser number of hidden layers and highest generalization power considered for pruning of its hidden neurons. Experiments have shown that the resultant architecture generated by this approach exhibits same or better performance as compared with the original network architecture.