- Poster presentation
- Open Access
- Published:
A cortical theory of super-efficient probabilistic inference based on sparse distributed representations
BMC Neuroscience volume 14, Article number: P324 (2013)
The remarkable structural homogeneity of isocortex strongly suggests a canonical cortical algorithm that performs the same essential function in all regions [1]. That function is widely construed/modeled as probabilistic inference, i.e., the ability, given an input, to retrieve the best-matching memory (or, most likely hypothesis) stored in memory. In [2], I described a cortical model for which both storage (learning) of new items into memory and probabilistic inference are constant time operations, which is a level of performance not present in any other published information processing system. This efficiency depends critically on: a) representing inputs with sparse distributed representations (SDRs), i.e., relatively small sets of binary units chosen from a large pool; and on b) choosing (learning) new SDRs so that more similar inputs are mapped to more highly intersecting SDRs. The macrocolumn (specifically, its pool of L2/3 pyramidals) was proposed as the large pool, with its minicolumns acting in winner-take-all fashion, ensuring that macrocolumnar codes consist of one winner per minicolumn. Here, I present results of large hierarchical model instances, having many levels and hundreds of macrocolumns, performing: a) single-trial learning of sets of sequences derived from natural video; and b) immediate (i.e., no search) retrieval of best-matching stored sequences. Figure 1 shows the major shift in going from the localist coding scheme present in most hierarchical cortical models, e.g., [3], to SDR coding. In Figure 1A, each feature in a coding module (red rectangle) is represented by a single unit, whereas in Figure 1B, each feature in a coding module (red hexagon) is represented by a set of co-active units, one per minicolumn. Yellow call-outs show a sample suggesting the potentially large number of other features stored in a macrocolumn. This change has a potentially large impact on explaining the storage capacity of cortex, but more importantly on explaining the speed and other characteristics of probabilistic/approximate reasoning possessed by biological brains.
References
Douglas RJ, Martin KA, Witteridge D: A canonical microcircuit for neocortex. Neural Computation. 1989, 1 (4): 480-488. 10.1162/neco.1989.1.4.480.
Rinkus GJ: A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality. Frontiers in Neuroanatomy. 2010, 4 (17): doi:10.3389/fnana.2010.00017
Giese MA, Poggio T: Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience. 2003, 4 (3): 179-192.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Rinkus, R. A cortical theory of super-efficient probabilistic inference based on sparse distributed representations. BMC Neurosci 14 (Suppl 1), P324 (2013). https://doi.org/10.1186/1471-2202-14-S1-P324
Published:
DOI: https://doi.org/10.1186/1471-2202-14-S1-P324
Keywords
- Hierarchical Model
- Code Scheme
- Information Processing System
- Large Pool
- Structural Homogeneity