Reservoir Computing with Ultra-Sparse Rings

Abstract

Many existing models of computation in recurrent neural networks assume dense, unconstrained initial connectivity, where any pair of neurons may be coupled to generate the rich dynamics needed for learning complex temporal patterns. Inspired by invertebrate circuits that often exhibit ring-like connectivity, we show that computation can occur in ultra-sparse spiking and rate reservoirs that are initially coupled as simple unidirectional rings. In contrast to standard recurrent networks, the total number of network parameters in these ring networks scales only linearly with network size, while still producing rich feature sets. We demonstrate that such networks can successfully reproduce a range of dynamical systems tasks, including oscillations, multi-stable switches, and low-dimensional chaotic attractors. Our findings show that structured spatio-temporal dynamics naturally arising from large ring topologies, often observed in invertebrate circuits, are a sufficient mechanism for learning different types of attractors.