When are microcircuits well-modeled by maximum entropy methods?

Recent experiments in retina and cortex have demonstrated that pairwise maximum entropy (PME) methods can approximate observed spiking patterns to a high degree of accuracy [1, 2]. In this paper we examine the relationship between network architecture, statistics of transmitted signals, and the order of correlation needed to explain network output in a class of simple circuits. We find that the pairwise maximum entropy fit of output spiking patterns is informative about signal statistics but not necessarily network architecture; in particular, the order of correlation cannot be used to make a common and intuitive inference about upstream projections.

Methods

We consider systems of three spiking cells, driven by a common fluctuating input against independent background noise. The cells are chosen to be either simple sum-and-threshold units, linear integrate and fire, or nonlinear (fitted to spiking characteristics of retinal ganglion cells) integrate and fire. The inputs are chosen from distributions that are either gaussian, uniform, skewed, or bimodal. For each circuit, we compute the distribution on output spiking states either analytically or by sampling, approximate by a maximum entropy fit, and measure the goodness of fit via the Kullback-Leibler divergence between the two. We repeat over a range of input parameters: mean, total input variance, and the relative strength of common drive to background noise.

Results

In analytical studies, we identified a simple "consistency" condition relating the probabilities of various network states that must hold if network outputs are to be perfectly described by a PME distribution. The consistency condition indeed is approximately satisfied, and the activity of this circuit is well-modeled by the PME fit, if the inputs are unimodal. This is in agreement with the results of [3, 4] in the gaussian case, but it extends beyond gaussian inputs. Indeed, this result holds regardless of whether spike generation is characterized by a simple static nonlinearity, or by integrate-and-fire (LIF) threshold-crossing dynamics. However, this circuit does generate substantial higher-order correlations if the common input to the three cells is bimodal. This approach complements the distinct expansions in [5], enabling instead a geometric picture of "regions" of network output statistics expected that will produce the strongest deviations from pairwise descriptions.

Finally we address the question of whether the goodness-of-fit of the PME approximation is informative about network architecture. We have already seen that high order inputs do not imply deviations from the PME fit. To examine the converse question, we study a simple circuit driven by pairwise bimodal inputs. We find thatthis circuit can generate significant deviations from the PME fit, despite the pairwise anatomical projections.