Volume 12 Supplement 1

Twentieth Annual Computational Neuroscience Meeting: CNS*2011

Open Access

Stochastic gradient ascent learning with spike timing dependent plasticity

BMC Neuroscience201112(Suppl 1):P250

DOI: 10.1186/1471-2202-12-S1-P250

Published: 18 July 2011

Stochastic gradient ascent learning exploits correlations of parameter variations with overall success of a system. This algorithmic idea has been related to neuronal network learning by postulating eligibility traces at synapses, which make them selectable for synaptic changes depending on later reward signals ([1] and [2]). Formalizations of the synaptic and neuronal dynamics supporting gradient ascent learning in terms of differential equations exhibit strong similarities with a recent formulation of spike timing dependent plasticity (STDP) [3] when it is combined with a reward signal. Here we present conditions under which reward modulated STDP is in fact guaranteed to maximize expected reward. We present numerical simulations underlining the relevance of realistic STDP models for reward dependent learning. In particular, we find that the nonlinear adaptation to pre- and post-synaptic activities of STDP [3] contributes to stable learning.
https://static-content.springer.com/image/art%3A10.1186%2F1471-2202-12-S1-P250/MediaObjects/12868_2011_Article_2268_Fig1_HTML.jpg
Figure 1

Learning the XOR function with a reward modulated STDP rule. Left: Output activity versus training episode in a feed forward network with Poisson-like neurons (2 input nodes, 10 hidden nodes and 1 output node). The output activity for the [true, false] and [false, true] inputs becomes stronger, while the output for the [true, true] and [false, false] inputs becomes weak after training. Right: Accumulated administered reward for the four input patterns versus training episode.

Authors’ Affiliations

(1)
Institute for Theoretical Physics, University of Bremen

References

  1. Sebastian Seung H: Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission. Neuron. 2003, 40 (6): 1063-1073. 10.1016/S0896-6273(03)00761-X.View ArticlePubMedGoogle Scholar
  2. Xiaohui Xie, Sebastian Seung H: Learning in neural networks by reinforcement of irregular spiking. Phys Rev E. 2004, 69 (4): 041909-1-041909-10.Google Scholar
  3. Schmiedt Joscha T, Christian Albers, Klaus Pawelzik: Spike timing-dependent plasticity as dynamic filter. Advances in Neural Information Processing Systems 23. Edited by: J. Lafferty and C. K. I. Williams and J. Shawe-Taylor and R.S. Zemel and A. Culotta. 2010, 2110-2118.Google Scholar

Copyright

© Vieira et al; licensee BioMed Central Ltd. 2011

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement