Dynamically deactivating hidden neurons in a multilayer perceptron neural network

Abstract

In this paper we present an approach that terminates the processing of hidden nodes, within a multilayer perceptron (MLP) neural network, if they become inactive during the learning process. The determination of the activity and non-activity of hidden nodes are based on the mean deviation of changes in the average derivative of the hidden nodes within an interval of several iterations. A decreasing threshold value is used to evaluate the mean deviation and hence to deactivate the hidden nodes accordingly.