General form of learning algorithms for neuromorphic hardware implementation

The DARPA Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE) initiative aims to create a new generation of high-density, low-power consumption chips capable of replicating adaptive and intelligent behavior observed in animals. To ensure fast speed, low power consumption, and parallel learning in billions of synapses, the learning laws that govern the adaptive behavior must be implemented in hardware. Over the past decades, multitudes of learning laws have been proposed in the literature to explain how neural activity shapes synaptic connections to support adaptive behavior. In order to implement as many of these laws as possible on the hardware, some general and easily parameterized form of learning law has to be designed and implemented on the chip. Such a general form would allow instantiation of multiple learning laws through different parameterizations without rewiring the hardware.

From the perspectives of usefulness, stability, homeostatic properties, and spatial and temporal locality, this project analyzes four categories of existing learning rules:

1. 1.

   Hebb rule derivatives with various methods for gating learning and decay;

2. 2.

   Threshold rule variations including the covariance and BCM families;

3. 3.

   Error-based learning rules; and

4. 4.

   Reinforcement rules

For each individual category a general form that can be implemented in hardware was derived. Even more general forms that include multiple categories are further suggested.