Perceptual Coherence : Hearing and Seeing

Information =−Σ. Pr(x i ) log 2 Pr(x i ). (3.3)
Therefore, events with low probabilities contribute little to the overall
information in spite of their high surprise value. The maximum information
and uncertainty occur when each stimulus has equal probability, and in that
case the information reduces to log 2 N, where N refers to the number of
stimuli. As the probability distribution becomes more unequal, the overall
information progressively decreases. If one stimulus occurs all of the time,
the information goes to 0. The measure of information is bits if using
log 2—it is only a number, like a percentage, without a physical dimension.
If we want to define the information for combinations of two variables
(x i , y j ), then the information summed over all xs and ys becomes
Information =−ΣΣPr(x i , y j ) log 2 Pr(x i , y j ). (3.4)
If the two variables are independent, then
Characteristics of Auditory and Visual Scenes 99
Information =−ΣΣPr(x i ) Pr(y j ) log 2 Pr(x i ) Pr(y j ) (3.5)
Information =−Σ. Pr(x i ) log 2 Pr(x i ) −ΣPr(y j ) log 2 Pr(y j ) (3.6)
Information = I(x) + I(y). (3.7)
But natural stimuli are never made up of independent (or random) variables,
and natural messages are never composed of independent random sequences
of elements. For example, the letter q is nearly always followed by
the letter u. After identifying a q, knowing that u occurrs next rarely gives us
any information (or conversely, that letter position does not create any uncertainty).
We measure the actual information in terms of conditional probabilities;
that is, given the letter q, what is the probability of u in the following
letter slot? The difference between the information based on independent
variables and that actually measured is the constraint or structure in the
stimulus or the mutual information between stimulus and response.
There are several important points about this measure of information:
1. The information measure can be generalized to variables that have continuous
distributions that often involve integrating over the range of the
values. Moreover, the information measure can be further generalized
to random functions such as sound pressure waveforms over time.
2. The information does not depend on the central tendency (i.e., mean)
of the (stimulus or response) distribution. The information does depend
on the number of possible states. If all states are equally probable, increasing
the number of states twofold increases the information by 1 bit.
3. As Rieke et al. (1997) pointed out, the number of states for continuous
variables is infinite. To make the measure valid for continuous variables,
we need to create discrete bins in which we cumulate the values of the
variable at discrete time points. Thus, our measurements will always