Skip to content

Advertisement

  • Poster presentation
  • Open Access

An actor-critic model of saccade adaptation

BMC Neuroscience201314 (Suppl 1) :P425

https://doi.org/10.1186/1471-2202-14-S1-P425

  • Published:

Keywords

  • Basal Ganglion
  • Ventral Tegmental Area
  • Reinforcement Learning
  • Supervise Learning
  • Motor Learning

The basal ganglia and the cerebellum are subcortical structures indispensable for voluntary motor control and motor learning. They are thought to perform reinforcement learning and supervised learning, respectively, and interact with each other [1]. Yet, how these structures and their learning mechanisms interact remains unknown.

In this study, we propose a model of interaction between the basal ganglia and the cerebellum for voluntary motor control and motor learning. We consider that the basal ganglia performs temporal difference (TD) learning. Specifically, according to the electrophysiological experiments [2], we assume that neurons in ventral tegmental area (VTA), a part of the basal ganglia, represent the value of delta, the prediction error of TD-learning. On the other hand, we consider that the cerebellum generates motor commands through supervised learning, for which the inferior olive (IO) provides teacher signals. Here, based on the anatomical findings of dopaminergic inputs from VTA to the IO [3], we assume that the cerebellum can receive the information of TD-prediction error as teacher signals via the IO. In the end, we propose a scheme of the interaction between the basal ganglia and the cerebellum as an actor-critic model in reinforcement learning (Figure 1A, [4]).
Figure 1
Figure 1

(A) Proposed scheme of interaction between the basal ganglia and the cerebellum as an actor-critic model. (B) Illustration of direction-adaptation of saccades.

We adopt the proposed scheme to double-step adaptation of saccade, which is voluntary eye movement and is mediated by a distributed network including both the basal ganglia and the cerebellum. A double-step saccade adaptation paradigm called direction adaptation goes as follows (Figure 1B). Initially, the eye is fixated at the center position. Next, a target appears at a certain position, and the eye moves to the target (first saccade). When the saccade starts, the target is immediately removed and reappears to another position. In turn, the eye moves to the second target (corrective saccade). By repeating many trials, when the first target appears, the eye moves to the position of the expected second target. Our proposed model reproduces this direction-adaptation of saccades. These results suggest that the interaction between the basal ganglia and the cerebellum as an actor-critic model provides a powerful motor control and learning mechanism.

Authors’ Affiliations

(1)
Graduate School of Informatics and Engineering, The University of Electro-Communications, 1-5-1 Chofugaoka, Chofu, Tokyo 182-8585, Japan

References

  1. Doya K: What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?. Neural Netw. 1999, 12: 961-974. 10.1016/S0893-6080(99)00046-5.View ArticlePubMedGoogle Scholar
  2. Shultz W: Predictive reward signal of dopamine neurons. J Neurophysiol. 1998, 80: 1-27.Google Scholar
  3. Winship IR, Pakan JM, Todd KG, Wong-Wylie DR: A comparison of ventral tegmental neurons projecting to optic flow regions of the inferior olive vs. the hippocampal formation. Neuroscience. 2006, 141: 463-473. 10.1016/j.neuroscience.2006.03.057.View ArticlePubMedGoogle Scholar
  4. Sutton RS, Barto AG: Reinforcement learning: An introduction. 1998, MIT PressGoogle Scholar

Copyright

© Inaba and Yamazaki; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement