Multi-modal novelty and familiarity detection

The neural network is built on integrate-and-fire spiking neurons with active dendrites (ADDS) and trained with STDP rule presented in [4]. The overall architecture is shown in Figure 1 (left). It integrates representation of concepts in different modalities. The distributed representation in the working memory area achieve temporal binding via phase-locking, whereas different concepts are separated in time. The robot implementation provides interpretation and execution of simple instructions given via the auditory area and interpreted in the working memory which is integrating sequential language input into instruction representation. The separate entities of the instruction and the instruction as a whole run in nested gamma-theta oscillations. The novelty and familiarity detectors network is shown on Figure 1 (right). The familiarity detector (FD) and novelty detector (ND) areas are 25x25 ADDS neurons trained with STDP. After training approximately 68% (± 15 in the different trials / training sessions) of the FD neurons become unimodal action/object specific. About 22% (±15) of the ND neurons were also observed to become unimodal action/object specific, responding to when a particular object/action are not present in the verbal instruction or the visual field.