Neural conduction delay forces the emergence of predictive function in simulated evolution
© Mann and Choe; licensee BioMed Central Ltd. 2010
Published: 20 July 2010
Evidence from biological studies suggests that humans are able to predict the sensory consequences of their own actions . Computational studies also demonstrate the advantage of systems that predict sensory consequences of actions over those that predict the value of actions alone . But how could the ability to predict sensory consequences of actions have evolved? One solution suggested by  is that prediction mechanisms first evolved to deal with natural sources of delay. Delay is commonly considered to be a purely negative feature of real world systems; however, we argue that delay can actually encourage evolution of the prediction of sensory consequences. We hypothesize that increasing sensory delay to an evolving population of sensory-motor agents will increase reliance on internal prediction of sensory consequences.
To test our hypothesis we evolved populations of artificial neural networks at a complex control task (i.e. pole balancing, see figure 1) with varied neural conduction delay (Δt) between sensory neurons and input to the control network (see figure 2), which estimates the long term cost of applying a specific action. For top fitness networks, hidden unit activations were recorded as well as the true consequent sensory state during several evaluation trials. Each sensory variable was associated with the hidden unit that the sensory variable was maximally correlated with. Taking the average of these correlation values provides a measure of how well an agent can predict the sensory consequences of actions. We expected to find that increasing sensory delay also increases the average correlation measure described above.
The result of the experiment (summarized in figure 3) show that with no delay successful agents use a range of strategies, however, as delay increases successful strategies are forced to rely more and more on prediction of the next state to compensate for sensory delay. This seems surprising when considering that under conditions of no delay it is considerably easier to predict the next state than conditions with increased delay.
- Bullock D, Grossberg S, Geunther F: A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm. Journal of Cognitive Neuroscience. 1993, 54: 408-435. 10.1162/jocn.19126.96.36.1998.View ArticleGoogle Scholar
- Kuvayev L, Sutton RS: Model-based reinforcement learning with an approximate, learned model. Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems. 1996, 101-105.Google Scholar
- Kwon J, Choe Y: Facilitating neural dynamics for delay compensation: A road to predictive neural dynamics?. Neural Networks. 2009, 22: 267-276. 10.1016/j.neunet.2009.03.011.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd.