A computational model of the basal ganglia as a rewarded activity selection circuit with non-specific output

Introduction

According to reinforcement learning models, the basal ganglia are capable of selecting motor programs based on rewards from prior actions. In many of the models in the literature, this requires specific channels running through the basal ganglia which correspond to individual motor programs [1, 2]. These channels must be maintained through each stage of the basal ganglia and back to the cortex. The projection from motor thalamus to motor cortex, however, is a type II thalamocortical projection – a diffuse projection not capable of relaying information with much spatial resolution.

Model

This work presents a modification of these models which takes into account the diffuse thalamocortical projection. According to this hypothesis, the cortex generates distinct patterns of activity which correspond to different motor programs. Each activity pattern inhibits other patterns and experiences fatigue. The result is that, under normal circumstances, these patterns "take turns" being active. Activation of the thalamocortical projection excites all of the cortical neurons, causing a change in the dynamics of the model, such that the active pattern remains dominant as long as the thalamocortical projection is active.

Simulation

To exhibit this theory, a highly-detailed model which matches anatomical and physiological observations has been created. This model employs models of individual neurons which match the observed behavior of actual neurons, including up- and down-states and delayed spiking in striatal neurons and burst and phasic firing in pallidal and subthalamic neurons. Network architecture, including synaptic connectivity and strength, relative number of neurons in the various nuclei, and synaptic plasticity rules was also based on physiological and anatomical data.

Results

The results of simulations of this model show that it is capable of reinforcement learning. It also reproduces important experimental observations on the basal ganglia, including the effects of dopamine depletion, neural activity in various nuclei, and the effects of cortical electrical stimulation. Additionally, during the simulation of this model, several unexpected dynamics were observed which can be tested experimentally.