Skip to main content

Advertisement

The attention-gated reinforcement learning model can explain experimentally observed changes in tuning curves that follow category learning

Article metrics

  • 1322 Accesses

A problem with many traditional neural network learning algorithms is that they lack a biologically plausible method of assigning credit to the nodes in the earlier levels of the network that played a decisive role in the stimulus response mapping. In the Attention-Gated Reinforcement Learning Model (AGREL) [1] the credit assignment problem is overcome by the focal influence of attention through the feedback connections in the network. In each trial, a global reward value (conveyed by neuromodulators) is calculated and serves to increase the likelihood that successful behavior be repeated. The activity in the output level of the network (which is limited so that only a single node at a time may be active/attended) is back propagated through the network via feedback connections. The feedback connections, which are reciprocal in strength to the feedforward connections, allow the network to selectively target the lower level nodes that drove the current output, which in combination with the global reward factor, allows the network to enhance connections with nodes that caused successful behaviors, see figure 1. This attentional gating of the reward signal offers a biologically realistic solution to the credit assignment problem and in so doing provides a coherent and unifying framework for learning, with attentional selection at its core. AGREL has previously been shown to closely replicate the changes in tuning curves observed in two physiological categorization tasks [1]. In the current study, we first replicate the findings of two further physiological tasks using AGREL, and second, examine the effect of removing the feedback connections, on our models performance in these and the previous tasks.

Figure 1
figure1

AGREL network architecture, and learning rules for a 3 layered network. δ = reward signal.

One of the modeled tasks was conducted by Freedman and Assad [2], where monkeys were trained to group 12 directions of coherently moving dots into 2 arbitrary categories. They found that neurons in region LIP learned to respond selectively to a particular category following training (see figure 2). This finding was replicated in AGREL using a three layer network, in which the units in the hidden layer where shown to exhibit similar categorization dependent tuning properties, see figure 2. In general, when the feedback connections were removed, the performance of the network was severely impaired, and the physiologically observed changes in tuning curves were no longer seen, see figure 2. However, depending on the complexity of the task, the network was still able to converge to a solution, although convergence times were now far longer.

Figure 2
figure2

Radial histogram plots, in which each neuron is assigned to the direction of its preferred boundary (i.e. max difference between responses on either side). 1st: taken from [2], 2nd: shows AGREL with feedback connections, and 3rd: shows AGREL without feedback connections.

References

  1. 1.

    Roelfsema P, van Ooyen A: Attention-gated reinforcement learning of internal representations for classification. Neural Computation. 2005, 17: 2176-2214. 10.1162/0899766054615699.

  2. 2.

    Freedman D, Assad J: Experience-dependent representation of visual categories in parietal cortex. Nature. 2006, 443: 85-88. 10.1038/nature05078.

Download references

Author information

Correspondence to Lawrence A Watling.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Watling, L.A., Geerts, R., Roelfsema, P.R. et al. The attention-gated reinforcement learning model can explain experimentally observed changes in tuning curves that follow category learning. BMC Neurosci 10, P137 (2009) doi:10.1186/1471-2202-10-S1-P137

Download citation

Keywords

  • Category Learning
  • Tuning Curve
  • Attentional Selection
  • Feedback Connection
  • Successful Behavior