On the overfitting of the five-layered bottleneck network

Abstract

In autoassociative learning for the bottleneck neural network, the problem of overfitting is pointed out. This overfitting is pathological in the sense that it does not disappear even if the sample size goes to infinity. However, it is not observed in the real learning process. Thus we study the basin of the overfitting solution. First, the existence of overfitting is confirmed. Then it is shown that the basin of the overfitting solution is small compared with the normal solution.