The beneficial effects of non-specific synaptic plasticity for pattern recognition in auto-associative memory
© Calcraft et al; licensee BioMed Central Ltd. 2011
Published: 18 July 2011
Most learning theories for artificial neural networks and real neural systems assume weight changes that are specific to activated synapses. However, experimental evidence suggests that different forms of synaptic plasticity such as long-term potentiation in hippocampal CA1 pyramidal cells  and long-term depression in cerebellar Purkinje cells  are not completely synapse specific, and also affect inactive synapses in the neighborhood of the activated ones. The functional role of this non-specific synaptic plasticity (NSSP) is not clear.
We explore in simulation the effect of NSSP during the training cycle of an associative memory, finding that when the presence of NSSP causes weight changes of nearby synapses to influence each other during training, the network becomes more resilient to performance degradation when patterns are shifted from their original training positions during attempted recall. Indeed, recall of shifted patterns was found to be better in networks with NSSP than in networks without.
A fully connected one-dimensional associative memory using perceptron training rules was trained on a set of unbiased patterns in which on or off nodes appear in small clusters*. The network was then tested on noisy versions of the original patterns, and a measure of network performance was taken. The test patterns were then shifted by one position relative to the training patterns before random noise was added, and performance measures were again taken. This was repeated with progressively greater shifts in the test patterns. As expected, it was found that the greater the shift in the test pattern with respect to the original, the more poorly the network performed.
However, when we altered the training rule to incorporate NSSP, so that each time a weight was adjusted during training, weights on nearby synapses** were also changed, unexpected results were seen. It was found that although in cases where the test pattern set had not been shifted, the network with NSSP performed less well than the network without it, in cases where the test pattern set had been shifted the network with NSSP performed better than the network without.
This suggests that the performance of associative memories required to recognize patterns that are not precisely aligned with patterns that the network has trained on could be improved when synapses are allowed to influence other synapses in their neighborhood during training, which indicates a potential functional role for NSSP in the brain.
*These tests were carried out with networks of both 100 and 500 nodes, and with patterns with a mean cluster length of 5 units.
**Varying amounts of leakage were used in the training routines, affecting 1, 2 ,3 ,4 or 5 of the synapses either side of the target synapse. In these tests we assume the 'ideal' case in which axons emanating from physically adjacent nodes in the network terminate at synapses which are also physically adjacent to one another.
- Engert F, Bonhoeffer T: Synapse specificity of long-term potentiation breaks down at short distances. Nature. 1997, 388: 279-84.View ArticlePubMedGoogle Scholar
- Wang SS, Khiroug L, Augustine GJ: Quantification of spread of cerebellar long-term depression with chemical two-photon uncaging of glutamate. Proc Natl Acad Sci USA. 2000, 97: 8635-40.PubMed CentralView ArticlePubMedGoogle Scholar
- Safaryan K, Maex R, Adams R, Davey N, Steuber V: The effect of non-specific LTD on pattern recognition in cerebellar Purkinje cells. BMC Neuroscience. 2010, 11 (Suppl 1): 118.View ArticleGoogle Scholar
- Calcraft L, Adams R, Davey N: The performance of sparsely-connected 2D associative memory models with non-random images. Progress in Neural Processing. 2009, 18: 103-113.Google Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.