Developing neural networks with neurons competing for survival

Abstract

We study developmental growth in a feedforward neural network model inspired by the survival principle in nature. Each neuron has to select its incoming connections in a way that allow it to fire, as neurons that are not able to fire over a period of time degenerate and die. In order to survive, neurons have to find reoccurring patterns in the activity of the neurons in the preceding layer, because each neuron requires more than one active input at any one time to have enough activation for firing. The sensory input at the lowest layer therefore provides the maximum amount of activation that all neurons compete for. The whole network grows dynamically over time depending on how many patterns can be found and how many neurons can maintain themselves accordingly. If a neuron has found a stable firing pattern, a new neuron is created in the same layer. It is also made sure that there is always at least one neuron in each activated layer that is searching for novel patterns. If a layer stops growing for a certain amount of time, a new layer is created starting with a single neuron.