- Poster presentation
- Open Access
A cortical theory of super-efficient probabilistic inference based on sparse distributed representations
BMC Neurosciencevolume 14, Article number: P324 (2013)
The remarkable structural homogeneity of isocortex strongly suggests a canonical cortical algorithm that performs the same essential function in all regions . That function is widely construed/modeled as probabilistic inference, i.e., the ability, given an input, to retrieve the best-matching memory (or, most likely hypothesis) stored in memory. In , I described a cortical model for which both storage (learning) of new items into memory and probabilistic inference are constant time operations, which is a level of performance not present in any other published information processing system. This efficiency depends critically on: a) representing inputs with sparse distributed representations (SDRs), i.e., relatively small sets of binary units chosen from a large pool; and on b) choosing (learning) new SDRs so that more similar inputs are mapped to more highly intersecting SDRs. The macrocolumn (specifically, its pool of L2/3 pyramidals) was proposed as the large pool, with its minicolumns acting in winner-take-all fashion, ensuring that macrocolumnar codes consist of one winner per minicolumn. Here, I present results of large hierarchical model instances, having many levels and hundreds of macrocolumns, performing: a) single-trial learning of sets of sequences derived from natural video; and b) immediate (i.e., no search) retrieval of best-matching stored sequences. Figure 1 shows the major shift in going from the localist coding scheme present in most hierarchical cortical models, e.g., , to SDR coding. In Figure 1A, each feature in a coding module (red rectangle) is represented by a single unit, whereas in Figure 1B, each feature in a coding module (red hexagon) is represented by a set of co-active units, one per minicolumn. Yellow call-outs show a sample suggesting the potentially large number of other features stored in a macrocolumn. This change has a potentially large impact on explaining the storage capacity of cortex, but more importantly on explaining the speed and other characteristics of probabilistic/approximate reasoning possessed by biological brains.
Douglas RJ, Martin KA, Witteridge D: A canonical microcircuit for neocortex. Neural Computation. 1989, 1 (4): 480-488. 10.1162/neco.1918.104.22.1680.
Rinkus GJ: A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality. Frontiers in Neuroanatomy. 2010, 4 (17): doi:10.3389/fnana.2010.00017
Giese MA, Poggio T: Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience. 2003, 4 (3): 179-192.