Self-organization of asymmetric associative networks
BMC Neuroscience volume 10, Article number: P340 (2009)
Associative networks serve as basic paradigms for memory retrieval [1–3] and have been used to describe properties of many neuronal structures [4, 5]. In contrast, learning via activity dependent adaptation of real synapses is still not well understood. In particular, the Hopfield Model  artificially enforces synapses to represent neuronal correlations according to the Hebbian rule that unrealistically leads to symmetric couplings, catastrophic forgetting when large amounts of patterns are to be stored and explosion of weights when the Hebbian rule is applied iteratively. Various modifications of the simple Hebbian rule have been proposed, as for instance the use of global information to control Hebbian contributions to the weight matrix  that prohibits catastrophic forgetting and weight explosion. In contrast, we present a local learning rule that in large networks of N neurons can stabilize more than 1.6 N binary non-sparse random patterns (see Figure 1). When applied in an on-line fashion, it leads to retention of the stack of recent patterns without attrition. The synaptic algorithm turns out to be consistent with spike-timing dependent plasticity as observed in hippocampus and cortex. In fact, it resembles the perceptron rule  by modifying a synapse only when the postsynaptic neuron changes its activity. We also find that the mutual interaction of network dynamics with weight changes confines synaptic strength to finite values despite the formal instability of the local synaptic learning rule. Taken together our work suggests that stabilization could provide a unifying principle of weight-activity co-evolution, leading to large storage capacity, on-line ability, generalization and extraction of higher order correlations which has testable implications for synaptic dynamics and cortical function.
Palm G: Neural Assemblies. An Alternative Approach to Artificial Intelligence. 1982, Springer, Berlin, Heidelberg, New York
Hopfield JJ: Neural networks and physical systems with emergent collective computational abilities. Proc NatL Acad Sci USA. 1982, 79: 2554-2558. 10.1073/pnas.79.8.2554.
Hertz JA, Krogh A, Palmer RG: Introduction to the Theory of Neural Computation. 1991, Addison Wesley
Myashita Y: Inferior temporal cortex: Where visual perception meets memory. Ann Rev Neurosci. 1993, 16: 245-263. 10.1146/annurev.ne.16.030193.001333.
Amit DJ, Brunel N, Tsodyks MV: Correlations of cortical Hebbian reverberations: theory versus experiment. J Neuroscience. 1994, 14: 6435-6445.
Blumenfeld B, Preminger S, Sagi D, Tsodyks M: Dynamics of memory representations in networks with novelty-facilitated synaptic plasticity. Neuron. 52: 383-394. 10.1016/j.neuron.2006.08.016.
Many thanks to Misha Tsodyks for encouraging discussions.
About this article
Cite this article
Albers, C., Pawelzik, K. Self-organization of asymmetric associative networks. BMC Neurosci 10 (Suppl 1), P340 (2009). https://doi.org/10.1186/1471-2202-10-S1-P340