A new error backpropagation learning algorithm for a layered neural network with nondifferentiable units

Abstract

This paper proposes a new error backpropagation method (DBP) for a three-layered neural network containing a nondifferentiable binary output unit. In contrast to the conventional simple perceptron, in which the teacher signal is given only to the output layer, in the DBP method the teacher signal is also given to the middle layer so that the output error is decreased. Consequently, it is possible in the DBP method to correct the coupling weights in both the lower layer and the upper layer. This makes it easy to construct a network composed only of binary output units, which results in high-speed operation and is suitable for hardware implementation. When the DBP method is applied to linearly inseparable tasks such as XORing, the learning performance is greatly improved compared to learning by the simple perceptron, and almost the same learning performance as the conventional BP is obtained. © 2007 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 90(5): 40– 49, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20318