Invariance of visual operations at the level of receptive fields

The brain is able to maintain a stable perception although the visual stimuli vary substantially on the retina due to geometric transformations and lightning variations in the environment. These transformations comprise (i) local scaling transformations caused by objects of different size and at different distances to the observer, (ii) locally linearized image deformations caused by variations in the viewing direction in relation to the object, (iii) locally linearized relative motions between the object and the observer and (iv) local multiplicative intensity transformations caused by illumination variations. Let us assume that receptive fields should be constructed by linear operations that are shift-invariant over space and/or space-time, with an additional requirement that receptive fields must not create new image structures at coarser scales that do not correspond to simplifications of corresponding structures at finer scales.

Given the above structural conditions, we derive idealized families of spatial and spatio-temporal receptive fields that satisfy these structural requirements by necessity, based on Gaussian kernels, Gaussian derivatives or closely related operators [3, 4]. We show that there are very close similarities between the receptive fields predicted from this theory and receptive fields found by cell recordings in biological vision, including (i) spatial on-center-off-surround and off-center-on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) space-time separable spatio-temporal receptive fields in the LGN and V1 and (iv) non-separable space-time tilted receptive fields in V1 [3, 4]. Indeed, from kernels predicted by this theory it is possible to generate receptive fields similar to all the basic types of monocular receptive fields reported by DeAngelis et al [2] in their survey of classical receptive fields.

By complementing such receptive field measurements with selection mechanisms over the parameters in the receptive field families, we show how true invariance of receptive field responses can be obtained under scaling transformations, affine transformations and Galilean transformations [4]. Thereby, the framework provides a mathematically well-founded and biologically plausible model for how basic invariance properties can be achieved already at the level of receptive fields. In this way, the presented theory supports invariant recognition of objects and events under variations in viewpoint, retinal size, object motion and illumination.