Extreme sensitivity of reservoir computing to small network disruptions
BMC Neuroscience volume 16, Article number: P256 (2015)
Recent computational models based on reservoir computing (RC) are gaining attention as plausible theories of cortical information processing. In these models, the activity of a recurrently connected population of neurons is sent to one or many read-out units through a linear transformation. These models can operate in a chaotic regime which has been proposed as a possible mechanism underlying sustained irregular activity observed in cortical areas [1, 2]. Furthermore, models based on RC replicate the neural dynamics involved in decision making , interval timing , and motor control . However, one biological constraint that has been overlooked in these models is their resistance to small connectivity perturbations such as failures in synaptic transmission, a phenomenon that occurs frequently in healthy circuits without causing any drastic functional changes. Here, we show that different implementations of RC display very little resistance to small synaptic disruptions and discuss the implications of such fragility for RC mechanisms that may be present in neural coding. With the FORCE  procedure, networks lost their ability to replicate a jagged sinusoidal signal after a single neuron was removed from the reservoir (Figure 1A). Networks with innate training  showed a similar effect on a timing task (Figure 1B). The lag in the timing and the noise in the output both increased monotonically as further neurons were removed (Figure 1C,D); networks reached random performance after ~1.5% of neurons were eliminated. After the suppression of a single neuron, the spectrum of the weight matrix was greatly disturbed and repeated trials displayed unreliable trajectories, as assessed with principal components analysis. When individual synapses were removed instead of neurons, networks reached random performance after ~0.5% of synapses from the reservoir were eliminated. While living neuronal circuits can withstand small synaptic disruptions without compromising task performance, our results suggest that such disruptions have a catastrophic impact on the behaviour of RC models. Retraining the read-out unit seems to be futile as it results as a completely new solution post retraining instead of a finer restructuration. These results cast doubt on the validity of a large class of models that claim to capture the neuronal mechanisms of cognitive and behavioral tasks.
Sussillo D, Abbott LF: Generating Coherent Patterns of Activity from Chaotic Neural Networks. Neuron. 2009, 63: 544-557.
Laje R, Buonomano DV: Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat Neurosci. 2013, 16: 925-933.
Barak O, Sussillo D, Romo R, Tsodyks M, Abbott LF: From fixed points to chaos: Three models of delayed discrimination. Progress in Neurobiology. 2013, 103: 214-222.
This research was funded by grants to J.P.T. from NSERC Discovery and CIHR operating funds.
About this article
Cite this article
Vincent-Lamarre, P., Lajoie, G. & Thivierge, JP. Extreme sensitivity of reservoir computing to small network disruptions. BMC Neurosci 16 (Suppl 1), P256 (2015). https://doi.org/10.1186/1471-2202-16-S1-P256