Transient Chaotic Dimensionality Expansion by Recurrent Networks

Abstract

Cortical neurons communicate with spikes, which are discrete events in time. Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks. We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of the definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos. The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary networks: We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks, classification performance peaks rapidly within one activation per participating neuron, demonstrating fast event-based computation that may be exploited by biological neural systems, for which we propose testable predictions.