Pseudo-potentiality maximization for improved interpretation and generalization in neural networks

Abstract

The present paper proposes a new type of information-theoretic method called “pseudo potentiality maximization”. The potentiality means neurons' ability to respond appropriately to as many situations as possible. For the first approximation, the potentiality is represented by the variance of neurons toward input patterns. Because difficulty exists to compute and control this potentiality, the pseudo-potentiality is introduced with a parameter to control the amount of potentiality. By controlling this parameter, the potentiality is easily increased or decreased. The method was applied to the well-known Australian credit data set. The experimental results showed that the lowest generalization errors were obtained by the present method. In addition, interpretable connection weights were obtained, similar to the regression coefficients of the logistic analysis.