Volume 14 Supplement 1

Abstracts from the Twenty Second Annual Computational Neuroscience Meeting: CNS*2013

Open Access

An actor-critic model of saccade adaptation

BMC Neuroscience201314(Suppl 1):P425

https://doi.org/10.1186/1471-2202-14-S1-P425

Published: 8 July 2013

The basal ganglia and the cerebellum are subcortical structures indispensable for voluntary motor control and motor learning. They are thought to perform reinforcement learning and supervised learning, respectively, and interact with each other [1]. Yet, how these structures and their learning mechanisms interact remains unknown.

In this study, we propose a model of interaction between the basal ganglia and the cerebellum for voluntary motor control and motor learning. We consider that the basal ganglia performs temporal difference (TD) learning. Specifically, according to the electrophysiological experiments [2], we assume that neurons in ventral tegmental area (VTA), a part of the basal ganglia, represent the value of delta, the prediction error of TD-learning. On the other hand, we consider that the cerebellum generates motor commands through supervised learning, for which the inferior olive (IO) provides teacher signals. Here, based on the anatomical findings of dopaminergic inputs from VTA to the IO [3], we assume that the cerebellum can receive the information of TD-prediction error as teacher signals via the IO. In the end, we propose a scheme of the interaction between the basal ganglia and the cerebellum as an actor-critic model in reinforcement learning (Figure 1A, [4]).
Figure 1

(A) Proposed scheme of interaction between the basal ganglia and the cerebellum as an actor-critic model. (B) Illustration of direction-adaptation of saccades.

We adopt the proposed scheme to double-step adaptation of saccade, which is voluntary eye movement and is mediated by a distributed network including both the basal ganglia and the cerebellum. A double-step saccade adaptation paradigm called direction adaptation goes as follows (Figure 1B). Initially, the eye is fixated at the center position. Next, a target appears at a certain position, and the eye moves to the target (first saccade). When the saccade starts, the target is immediately removed and reappears to another position. In turn, the eye moves to the second target (corrective saccade). By repeating many trials, when the first target appears, the eye moves to the position of the expected second target. Our proposed model reproduces this direction-adaptation of saccades. These results suggest that the interaction between the basal ganglia and the cerebellum as an actor-critic model provides a powerful motor control and learning mechanism.

Authors’ Affiliations

(1)
Graduate School of Informatics and Engineering, The University of Electro-Communications

References

  1. Doya K: What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?. Neural Netw. 1999, 12: 961-974. 10.1016/S0893-6080(99)00046-5.View ArticlePubMedGoogle Scholar
  2. Shultz W: Predictive reward signal of dopamine neurons. J Neurophysiol. 1998, 80: 1-27.Google Scholar
  3. Winship IR, Pakan JM, Todd KG, Wong-Wylie DR: A comparison of ventral tegmental neurons projecting to optic flow regions of the inferior olive vs. the hippocampal formation. Neuroscience. 2006, 141: 463-473. 10.1016/j.neuroscience.2006.03.057.View ArticlePubMedGoogle Scholar
  4. Sutton RS, Barto AG: Reinforcement learning: An introduction. 1998, MIT PressGoogle Scholar

Copyright

© Inaba and Yamazaki; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement