The first five percent of neural contrast spikes from a visual scene robustly computes the gravity vector

Current approaches to determine upright orientation and rotation rely on inertial accelerometers combined with gravity-measuring sensors. Biological posture control uses three senses. The most well known sense is the inner ear, which senses angular rotation and linear velocity. Proprioception is the sensing of body posture and center of mass with respect to gravity. A third sense of balance and orientation information comes from the optical system. This presentation presents a biologically-inspired algorithm for determining absolute orientation and rotation using computer image processing. The approach relies on the statistical property of man-made or cultural environments to exhibit predominately more horizontal and vertical edges than oblique edges [1]. That is, vertical edges of cultural objects tend to mirror the gravity vector. A biologically-inspired Gaussian on-/off-center contrast model [2] is used to extract edges, the first N% of luminance contrast-based neural spikes (top N% of the brightest edge pixels) in various scales form a basis set for image segmentation, multiple regions of interest are summed to accumulate image statistics, and a line template is correlated to determine a probabilistic orientation distribution with respect to a body-centered reference frame. The algorithm was tested against 600 real-world, cultural images and achieved 96% correct upright orientation +/-0.5^o with a mean signal to noise ratio (SNR) of 10.75dB and 2.93dB standard deviation (see Figure 1). A sensitivity analysis was performed to determine the temporal percent of neural contrast spikes required for robust performance. It was found that as few as the first 5% of neural contrast spikes from a visual scene robustly (94%) computes the gravity vector, without prior knowledge of or training on the environment. Further, it was found that the quality of the accumulated image correlates to the probability of organic or natural scenes (P=0.0 for a point cloud) and to man-made or cultural scenes (P=1.0 for strong horizontal and vertical lines).

Sensitivity analysis wherein the horizontal axis is the first N% of accumulated luminance contrast-based neural spikes varying from 0.01% to 15%, for a constant-scale Gaussian on-/off-center contrast model. Note the images to the right shows the algorithm response is a point-cloud for organic scenes (bottom right) and strong vertical and horizontal lines for man-made scenes (top right). The algorithm was tested against 312 real-world, cultural images. The vertical axis plots the rotation angle percent correct for an upright orientation +/-0.5^o. Three insert plots show the SNR over the run set for the temporal first 0.1%, 5%, and 12% of neural contrast spikes.

Full size image