Perceptual Coherence : Hearing and Seeing

130 Perceptual Coherence
K
Figure 3.9. Kurtosis increases for sparse coding. Essentially, each output neuron is
either on or off; the cells rarely fire at an intermediate rate. Compared to a Gaussian
function (kurtosis = 0), the exponential function (kurtosis = 3) has a higher probability
of not firing at all, a lower probability of firing at low rates, and a higher probability
of firing at high rates.
The consequence of such sparse distributed coding is to change the activity
of individual cells from a theoretical normal distribution. Within any
time interval, a cell will fire at very high rates only for a small number of
inputs; the cell will not fire at all or will fire at a low rate for the vast majority
of inputs. Field and coworkers described this change in terms of kurtosis,
the peakedness (fourth moment) around the mean value. A comparison
between a uniform distribution, a normal distribution, and an exponential
distribution with a high value of kurtosis is shown in figure 3.9. As the
kurtosis increases, the uncertainty decreases and the redundancy increases
because the no-response output becomes more and more probable (the
maximum information transmission occurs when all outputs are equally
likely, the uniform distribution).
Sparse coding representations have several appealing features:
K
1. Sparse representations can be made relatively resistant to neurological
damage simply by duplicating the set of units responding to each
input (Foldiak & Young, 1995).
2. Sparse representations seem to match the appearance of natural images
in terms of a set of independent features such as edges that contain
correlations among sets of points (Barlow, 2001). By explicitly