Skip to content


  • Poster presentation
  • Open Access

A cortical theory of super-efficient probabilistic inference based on sparse distributed representations

BMC Neuroscience201314 (Suppl 1) :P324

  • Published:


  • Hierarchical Model
  • Code Scheme
  • Information Processing System
  • Large Pool
  • Structural Homogeneity
The remarkable structural homogeneity of isocortex strongly suggests a canonical cortical algorithm that performs the same essential function in all regions [1]. That function is widely construed/modeled as probabilistic inference, i.e., the ability, given an input, to retrieve the best-matching memory (or, most likely hypothesis) stored in memory. In [2], I described a cortical model for which both storage (learning) of new items into memory and probabilistic inference are constant time operations, which is a level of performance not present in any other published information processing system. This efficiency depends critically on: a) representing inputs with sparse distributed representations (SDRs), i.e., relatively small sets of binary units chosen from a large pool; and on b) choosing (learning) new SDRs so that more similar inputs are mapped to more highly intersecting SDRs. The macrocolumn (specifically, its pool of L2/3 pyramidals) was proposed as the large pool, with its minicolumns acting in winner-take-all fashion, ensuring that macrocolumnar codes consist of one winner per minicolumn. Here, I present results of large hierarchical model instances, having many levels and hundreds of macrocolumns, performing: a) single-trial learning of sets of sequences derived from natural video; and b) immediate (i.e., no search) retrieval of best-matching stored sequences. Figure 1 shows the major shift in going from the localist coding scheme present in most hierarchical cortical models, e.g., [3], to SDR coding. In Figure 1A, each feature in a coding module (red rectangle) is represented by a single unit, whereas in Figure 1B, each feature in a coding module (red hexagon) is represented by a set of co-active units, one per minicolumn. Yellow call-outs show a sample suggesting the potentially large number of other features stored in a macrocolumn. This change has a potentially large impact on explaining the storage capacity of cortex, but more importantly on explaining the speed and other characteristics of probabilistic/approximate reasoning possessed by biological brains.
Figure 1
Figure 1

Comparison of localist (A) and SDR-based (B) versions of visual feature hierarchies.

Authors’ Affiliations

Neurithmic Systems, 275 Grove St., Suite 2-400, Newton, Mass, 02466, USA


  1. Douglas RJ, Martin KA, Witteridge D: A canonical microcircuit for neocortex. Neural Computation. 1989, 1 (4): 480-488. 10.1162/neco.1989.1.4.480.View ArticleGoogle Scholar
  2. Rinkus GJ: A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality. Frontiers in Neuroanatomy. 2010, 4 (17): doi:10.3389/fnana.2010.00017Google Scholar
  3. Giese MA, Poggio T: Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience. 2003, 4 (3): 179-192.View ArticlePubMedGoogle Scholar