Skip to main content
  • Poster presentation
  • Open access
  • Published:

Exploring the link between temporal difference learning and spike-timing-dependent plasticity

Introduction

It has been shown that a form of temporal difference (TD) learning for predicting the value of the membrane potential of a neuron, at a fixed delay after the neuron received a presynaptic spike, results in a learning rule that is very similar to hebbian spike-timing-dependent plasticity (STDP) [1]. Since this result was obtained using a relatively complex neural model (two-compartmental, with Hodgkin-Huxley-like currents) and a simple setup (a single presynaptic spike followed by a single current pulse), we investigated whether it holds for simpler neural models and for more general situations. This is relevant both for the theoretical understanding of this phenomenon and for verifying that it holds in common simulations of spiking neural networks.

Methods

We studied this phenomenon using both integrate-and-fire (IAF) and Izhikevich neurons through simulation and for the IAF neuron, also analytically. Postsynaptic spikes were generated by single current pulses (as in the original study), by the irregular firing of synaptic afferents, or by a constant input current.

Results

For the IAF neuron, we found a hebbian, STDP-like plasticity rule only when postsynaptic spikes were generated by a single current pulse and the reset potential of the neuron was positive. For the same input and negative reset potential, as well as for constant input current (regardless of the reset potential), the resulting plasticity rule was anti-hebbian.

For the Izhikevich neuron we obtained hebbian STDP-like plasticity for both constant and pulsed input current. There is a qualitative difference with respect to the IAF case because the Izhikevich neuron incorporates the dynamics of the membrane potential during the onset of the action potential. By adding an action potential of non-zero duration to the IAF model, the shape of the plasticity function changes significantly and becomes similar to hebbian STDP. This shows that the plasticity function resulted from TD learning depends critically on whether the neuron adapts its synapses to learn the shape of its action potential or not. However, the shape of the action potential is commonly considered not to carry information. When we consider just the TD learning of the sub-threshold dynamics of the membrane potential, the shape of the resulted learning function can loose its similarity with hebbian STDP.

For both neural models, in the case of irregular synaptic input there was no clear relationship between the plastic changes predicted by TD learning and the temporal delay between the pre- and postsynaptic spikes. Moreover, the sign of these plastic changes did not depend uniquely on the sign of the temporal delay.

Conclusion

In conclusion, TD learning in spiking neurons does not always lead to hebbian STDP.

References

  1. Rao RPN, Sejnowski TJ: Spike-timing-dependent hebbian plasticity as temporal difference learning. Neural Comput. 2001, 13: 2221-2237. 10.1162/089976601750541787.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This work has been sponsored by a grant of the Romanian National Authority for Scientific Research (PNCDI II, Parteneriate, contract no. 11-039/2007).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Cătălin V Rusu.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Rusu, C.V., Florian, R.V. Exploring the link between temporal difference learning and spike-timing-dependent plasticity. BMC Neurosci 10 (Suppl 1), P201 (2009). https://doi.org/10.1186/1471-2202-10-S1-P201

Download citation

  • Published:

  • DOI: https://doi.org/10.1186/1471-2202-10-S1-P201

Keywords