Neural conduction delay forces the emergence of predictive function in simulated evolution

Evidence from biological studies suggests that humans are able to predict the sensory consequences of their own actions [1]. Computational studies also demonstrate the advantage of systems that predict sensory consequences of actions over those that predict the value of actions alone [2]. But how could the ability to predict sensory consequences of actions have evolved? One solution suggested by [3] is that prediction mechanisms first evolved to deal with natural sources of delay. Delay is commonly considered to be a purely negative feature of real world systems; however, we argue that delay can actually encourage evolution of the prediction of sensory consequences. We hypothesize that increasing sensory delay to an evolving population of sensory-motor agents will increase reliance on internal prediction of sensory consequences.

To test our hypothesis we evolved populations of artificial neural networks at a complex control task (i.e. pole balancing, see figure 1) with varied neural conduction delay (Δt) between sensory neurons and input to the control network (see figure 2), which estimates the long term cost of applying a specific action. For top fitness networks, hidden unit activations were recorded as well as the true consequent sensory state during several evaluation trials. Each sensory variable was associated with the hidden unit that the sensory variable was maximally correlated with. Taking the average of these correlation values provides a measure of how well an agent can predict the sensory consequences of actions. We expected to find that increasing sensory delay also increases the average correlation measure described above.

The result of the experiment (summarized in figure 3) show that with no delay successful agents use a range of strategies, however, as delay increases successful strategies are forced to rely more and more on prediction of the next state to compensate for sensory delay. This seems surprising when considering that under conditions of no delay it is considerably easier to predict the next state than conditions with increased delay.

Although the common conception of delay is negative, sensory delay can direct natural selection to favor individuals that are better able to predict the sensory consequences of their actions.

Cart-Pole Balancing

Full size image

Control network structure

Full size image

Absolute correlation between hidden until activations and variables of the state at time t+Δt as delay increases.

Full size image