An energy-efficient process of non-deterministic computation drives the emergence of predictive models and exploratory behavior

Abstract

Cortical neural networks encode information about the environment, combining data across sensory modalities to form predictive models of the world, which in turn drive behavioral output. Cortical population coding is probabilistic, with synchronous firing across the neural network achieved in the context of noisy inputs. The system-wide computational process, which encodes the likely state of the local environment, is achieved at a cost of only 20 Watts, indicating a deep connection between neuronal information processing and energy-efficient computation. This report presents a new framework for modeling non-deterministic computation in cortical neural networks, in terms of thermodynamic laws. Initially, free energy is expended to produce von Neumann entropy, then predictive value is extracted from that thermodynamic quantity of information. The extraction of predictive value during a single computation yields a percept, or a predictive semantical statement about the local environment, and the integration of sequential neural network states yields a temporal sequence of percepts, or a predictive syntactical statement about the cause-effect relationship between perceived events. The amount of predictive value available for computation is limited by the total amount of energy entering the system, and will always be incomplete, due to thermodynamic constraints. This process of thermodynamic computation naturally produces a rival energetic cost function, which minimizes energy expenditure: the system can either explore its local environment to gain potential predictive value, or it can exploit previously-acquired predictive value by triggering a contextually-relevant and thermodynamically-favored sequence of neural network states. The system grows into a more ordered state over time, as it physically encodes the predictive value acquired by interacting with its environment.