A computational model of the basal ganglia as a rewarded activity selection circuit with non-specific output
BMC Neuroscience volume 9, Article number: P61 (2008)
According to reinforcement learning models, the basal ganglia are capable of selecting motor programs based on rewards from prior actions. In many of the models in the literature, this requires specific channels running through the basal ganglia which correspond to individual motor programs [1, 2]. These channels must be maintained through each stage of the basal ganglia and back to the cortex. The projection from motor thalamus to motor cortex, however, is a type II thalamocortical projection – a diffuse projection not capable of relaying information with much spatial resolution.
This work presents a modification of these models which takes into account the diffuse thalamocortical projection. According to this hypothesis, the cortex generates distinct patterns of activity which correspond to different motor programs. Each activity pattern inhibits other patterns and experiences fatigue. The result is that, under normal circumstances, these patterns "take turns" being active. Activation of the thalamocortical projection excites all of the cortical neurons, causing a change in the dynamics of the model, such that the active pattern remains dominant as long as the thalamocortical projection is active.
To exhibit this theory, a highly-detailed model which matches anatomical and physiological observations has been created. This model employs models of individual neurons which match the observed behavior of actual neurons, including up- and down-states and delayed spiking in striatal neurons and burst and phasic firing in pallidal and subthalamic neurons. Network architecture, including synaptic connectivity and strength, relative number of neurons in the various nuclei, and synaptic plasticity rules was also based on physiological and anatomical data.
The results of simulations of this model show that it is capable of reinforcement learning. It also reproduces important experimental observations on the basal ganglia, including the effects of dopamine depletion, neural activity in various nuclei, and the effects of cortical electrical stimulation. Additionally, during the simulation of this model, several unexpected dynamics were observed which can be tested experimentally.
Frank MT: Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience. 2005, 17 (1): 51-72. 10.1162/0898929052880093.
Leblois A, Boraud T, Meissner W, Bergman H, Hansel D: Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia. The Journal of Neuroscience. 2006, 26 (13): 3567-3583. 10.1523/JNEUROSCI.5050-05.2006.
About this article
Cite this article
Hunn, D.R., Helms Tillery, S.I. A computational model of the basal ganglia as a rewarded activity selection circuit with non-specific output. BMC Neurosci 9 (Suppl 1), P61 (2008). https://doi.org/10.1186/1471-2202-9-S1-P61
- Basal Ganglion
- Reinforcement Learning
- Motor Program
- Striatal Neuron
- Dopamine Depletion