Emergence of behavioral primitives in self-organizing control and composition of behavior for autonomous robots

Background

Autonomous robots as well as animals process sensory information for the purpose of generating behaviors that are adapted to their respective environments. This includes the selection of behaviorally relevant perceptual features, the adaptation of control mechanisms and the storage and the recall of behavioral episodes for planning and execution. A robot as model of animal behavior should achieve sensorimotor control without being specifically programmed, but by exploring the behaviors that arise from the physical interaction between its body and the environment.

Self-organizing behavior

Efficient behavioral exploration can be obtained by the homeokinetic principle [1], a dynamical systems approach to robot control that establishes a self-tuned balance between sensitivity of actions to sensory inputs and predictability of the perceptual consequences of actions. The principle is effective in training artificial motor neurons to generate coherent movements that are suitable to explore the behavioral manifold [2]. Simultaneously, internal representations of the robot dynamics are learned by associative memory networks in the robot, which then play the role of an efference copy.

Composing behavioral primitives

In the present contribution we introduce an agent-based approach where a number of internal expert networks compete for the correct prediction of the control actions of a homeokinetic controller. As a result each agent achieves control over a specific reproducible behavior that forms a behavioral primitive. The obtained collection of behaviors is specific for the robot and the environment, but the elementary behaviors can be selected and combined to control the robot successful also in new environments. The composition of behaviors is achieved by a standard reinforcement learning algorithm with intrinsically determined rewards. The composition of the primitives is shown to be efficient for set of experts, but cannot be achieved when low-level actions used as elements. Videos of various robots (Figure 1) illustrate the study that also includes different degrees of specificity of the internal representations (Figure 2) and comparisons with classical methods.

Figure 1

Simulated spherical robot actuated by three internal masses that are moveable along their axes.

Figure 2

Partition of the space of angular velocities of the spherical robot by a set of control agents.

In conclusion, exploration of behavioral spaces by self-organizing control leads to the emergence of behavioral primitives that can be composed in order to generate complex goal-oriented behavior and can be used in motor planning.

Acknowledgment

The project was supported by the BCCN Göttingen grant #01GQ0432.