Skip to main content

Advertisement

An actor-critic model of saccade adaptation

Article metrics

  • 806 Accesses

The basal ganglia and the cerebellum are subcortical structures indispensable for voluntary motor control and motor learning. They are thought to perform reinforcement learning and supervised learning, respectively, and interact with each other [1]. Yet, how these structures and their learning mechanisms interact remains unknown.

In this study, we propose a model of interaction between the basal ganglia and the cerebellum for voluntary motor control and motor learning. We consider that the basal ganglia performs temporal difference (TD) learning. Specifically, according to the electrophysiological experiments [2], we assume that neurons in ventral tegmental area (VTA), a part of the basal ganglia, represent the value of delta, the prediction error of TD-learning. On the other hand, we consider that the cerebellum generates motor commands through supervised learning, for which the inferior olive (IO) provides teacher signals. Here, based on the anatomical findings of dopaminergic inputs from VTA to the IO [3], we assume that the cerebellum can receive the information of TD-prediction error as teacher signals via the IO. In the end, we propose a scheme of the interaction between the basal ganglia and the cerebellum as an actor-critic model in reinforcement learning (Figure 1A, [4]).

Figure 1
figure1

(A) Proposed scheme of interaction between the basal ganglia and the cerebellum as an actor-critic model. (B) Illustration of direction-adaptation of saccades.

We adopt the proposed scheme to double-step adaptation of saccade, which is voluntary eye movement and is mediated by a distributed network including both the basal ganglia and the cerebellum. A double-step saccade adaptation paradigm called direction adaptation goes as follows (Figure 1B). Initially, the eye is fixated at the center position. Next, a target appears at a certain position, and the eye moves to the target (first saccade). When the saccade starts, the target is immediately removed and reappears to another position. In turn, the eye moves to the second target (corrective saccade). By repeating many trials, when the first target appears, the eye moves to the position of the expected second target. Our proposed model reproduces this direction-adaptation of saccades. These results suggest that the interaction between the basal ganglia and the cerebellum as an actor-critic model provides a powerful motor control and learning mechanism.

References

  1. 1.

    Doya K: What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?. Neural Netw. 1999, 12: 961-974. 10.1016/S0893-6080(99)00046-5.

  2. 2.

    Shultz W: Predictive reward signal of dopamine neurons. J Neurophysiol. 1998, 80: 1-27.

  3. 3.

    Winship IR, Pakan JM, Todd KG, Wong-Wylie DR: A comparison of ventral tegmental neurons projecting to optic flow regions of the inferior olive vs. the hippocampal formation. Neuroscience. 2006, 141: 463-473. 10.1016/j.neuroscience.2006.03.057.

  4. 4.

    Sutton RS, Barto AG: Reinforcement learning: An introduction. 1998, MIT Press

Download references

Author information

Correspondence to Manabu Inaba.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Inaba, M., Yamazaki, T. An actor-critic model of saccade adaptation. BMC Neurosci 14, P425 (2013) doi:10.1186/1471-2202-14-S1-P425

Download citation

Keywords

  • Basal Ganglion
  • Ventral Tegmental Area
  • Reinforcement Learning
  • Supervise Learning
  • Motor Learning