'Entropy production rate' and 'entropy' for neural networks

Abstract

Two new quantities for neural networks, entropy production rate and entropy, are derived. In the Hopfield neural model, Hopfield introduced a quantity, energy, and the energy minimum corresponds to a possible good solution to a problem. It is shown that the energy function does not match the physical meaning of energy in physics; a better physical interpretation can go through entropy production rate and entropy in physics. These new quantities can be further extended to general nonequilibrium open systems for neural networks.<>