Self-organization of asymmetric associative networks

Introduction

Associative networks serve as basic paradigms for memory retrieval [1–3] and have been used to describe properties of many neuronal structures [4, 5]. In contrast, learning via activity dependent adaptation of real synapses is still not well understood. In particular, the Hopfield Model [2] artificially enforces synapses to represent neuronal correlations according to the Hebbian rule that unrealistically leads to symmetric couplings, catastrophic forgetting when large amounts of patterns are to be stored and explosion of weights when the Hebbian rule is applied iteratively. Various modifications of the simple Hebbian rule have been proposed, as for instance the use of global information to control Hebbian contributions to the weight matrix [6] that prohibits catastrophic forgetting and weight explosion. In contrast, we present a local learning rule that in large networks of N neurons can stabilize more than 1.6 N binary non-sparse random patterns (see Figure 1). When applied in an on-line fashion, it leads to retention of the stack of recent patterns without attrition. The synaptic algorithm turns out to be consistent with spike-timing dependent plasticity as observed in hippocampus and cortex. In fact, it resembles the perceptron rule [3] by modifying a synapse only when the postsynaptic neuron changes its activity. We also find that the mutual interaction of network dynamics with weight changes confines synaptic strength to finite values despite the formal instability of the local synaptic learning rule. Taken together our work suggests that stabilization could provide a unifying principle of weight-activity co-evolution, leading to large storage capacity, on-line ability, generalization and extraction of higher order correlations which has testable implications for synaptic dynamics and cortical function.

Figure 1

Retrieval of stored patterns depending on the relative number P / N of patterns learned in a network of N neurons.