Skip to content


  • Poster Presentation
  • Open Access

When are microcircuits well-modeled by maximum entropy methods?

  • 1Email author,
  • 1,
  • 2, 3 and
  • 4
BMC Neuroscience201011 (Suppl 1) :P65

  • Published:


  • Network Architecture
  • Retinal Ganglion Cell
  • Network Output
  • Maximum Entropy Method
  • Simple Circuit

Recent experiments in retina and cortex have demonstrated that pairwise maximum entropy (PME) methods can approximate observed spiking patterns to a high degree of accuracy [1, 2]. In this paper we examine the relationship between network architecture, statistics of transmitted signals, and the order of correlation needed to explain network output in a class of simple circuits. We find that the pairwise maximum entropy fit of output spiking patterns is informative about signal statistics but not necessarily network architecture; in particular, the order of correlation cannot be used to make a common and intuitive inference about upstream projections.


We consider systems of three spiking cells, driven by a common fluctuating input against independent background noise. The cells are chosen to be either simple sum-and-threshold units, linear integrate and fire, or nonlinear (fitted to spiking characteristics of retinal ganglion cells) integrate and fire. The inputs are chosen from distributions that are either gaussian, uniform, skewed, or bimodal. For each circuit, we compute the distribution on output spiking states either analytically or by sampling, approximate by a maximum entropy fit, and measure the goodness of fit via the Kullback-Leibler divergence between the two. We repeat over a range of input parameters: mean, total input variance, and the relative strength of common drive to background noise.


In analytical studies, we identified a simple "consistency" condition relating the probabilities of various network states that must hold if network outputs are to be perfectly described by a PME distribution. The consistency condition indeed is approximately satisfied, and the activity of this circuit is well-modeled by the PME fit, if the inputs are unimodal. This is in agreement with the results of [3, 4] in the gaussian case, but it extends beyond gaussian inputs. Indeed, this result holds regardless of whether spike generation is characterized by a simple static nonlinearity, or by integrate-and-fire (LIF) threshold-crossing dynamics. However, this circuit does generate substantial higher-order correlations if the common input to the three cells is bimodal. This approach complements the distinct expansions in [5], enabling instead a geometric picture of "regions" of network output statistics expected that will produce the strongest deviations from pairwise descriptions.

Finally we address the question of whether the goodness-of-fit of the PME approximation is informative about network architecture. We have already seen that high order inputs do not imply deviations from the PME fit. To examine the converse question, we study a simple circuit driven by pairwise bimodal inputs. We find thatthis circuit can generate significant deviations from the PME fit, despite the pairwise anatomical projections.



ESB acknowledges the support of the Burroughs-Wellcome Scientific Interfaces Program and NSF DMS #0818153. FMR acknowledges the support of HHMI and NEI/NIH grant EY11850.

Authors’ Affiliations

Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA
Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA
Howard Hughes Medical Institute, Seattle, WA 98195, USA
Department of Applied Mathematics and Theoretical Physics, Cambridge University, Cambridge, CB3 0WA, UK


  1. Schneidman E, Berry MJ, Segev R, Bialek W: Weak pairwise correlations imply strongly correlated network states in a neural population. Nature. 2006, 440: 1007-1012. 10.1038/nature04701.PubMed CentralView ArticlePubMedGoogle Scholar
  2. Shlens J, Field GD, Gauthier JL, Grivich MI, Petrusca D, Sher A, Litke AM, Chichilnisky EJ: The Structure of Multi-Neuron Firing Patterns in Primate Retina. Journal of Neuroscience. 2006, 26: 8254-8266. 10.1523/JNEUROSCI.1282-06.2006.View ArticlePubMedGoogle Scholar
  3. Macke JH, Berens P, Ecker AS, Tolias AS, Bethge M: Generating Spike Trains with Specified Correlation Coefficients. Neural Computation. 2009, 21: 397-423. 10.1162/neco.2008.02-08-713.View ArticlePubMedGoogle Scholar
  4. Bethge M, Berens P: Near-Maximum Entropy Models for Binary Neural Representations of Natural Images. Advances in Neural Information Processing Systems. 2008, Cambridge: MIT Press, 20: 97-104.Google Scholar
  5. Roudi Y, Nirenberg S, Latham PE: Pairwise Maximum Entropy Models for Studying Large Biological Systems, When They Can Work and When They Can’t. PLoS Computational Biology. 2009, 5: e1000380-10.1371/journal.pcbi.1000380.PubMed CentralView ArticlePubMedGoogle Scholar


© Barreiro et al; licensee BioMed Central Ltd. 2010

This article is published under license to BioMed Central Ltd.