Neuromodulation by surprise: a biologically plausible model of the learning rate dynamics
© Faraji et al; licensee BioMed Central Ltd. 2014
Published: 21 July 2014
Surprise is a central concept in learning, attention and the study of the neural basis of behaviour. However, how surprise affects learning and more specifically, how surprise affects synaptic learning rules in neural networks is largely undetermined. Here we study how surprise facilitates learning in different environments and how surprise can potentially modulate Hebbian learning in the form of a global factor in multi-factor learning rules.
Our proposed model explains how the agent should effectively control the speed of learning in different environments such that it matches both theory and empirical evidences from human and animal subjects. This model explains why surprising events provoke humans and animals to learn faster and why they rapidly adapt to changing environments. It also addresses the question of what the effective learning rate should be in both stable (either low-risky or high-risky) and volatile environments. Here effectiveness is defined as having a higher accuracy in learning a task, for instance the estimation of the mean reward in classic reinforcement learning, for a given time and computational complexity as well as the available memory as our constraints. This study also suggests a functional connectivity pattern for the neurochemical systems that are related to contextual modulation of learning rate. Further, it explains why we need different neuromodulators with distinct functional roles to act in parallel in a broad range of distribution and proposes suitable candidates responsible for measuring different quantities we need in the model.
This research was supported by the European Research Council (grant agreement no. 268 689, H.S.).
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.