Hybrid nets with variable parameters: a novel approach to fast learning under backpropagation

Abstract

This paper presents a novel approach under regular backpropagation. We introduce hybrid neural nets that have different activation functions for different layers in fully connected feed forward neural nets. We change the parameters of activation functions in hidden layers and output layer to accelerate the learning speed and to reduce the oscillation respectively. Results on the two-spirals benchmark are presented which are better than any results under backpropagation feed forward nets using monotone activation functions published previously.<>