Perceptual Coherence : Hearing and Seeing

358 Perceptual Coherence
for each instrument. The results indicated that listeners were quite accurate,
roughly 85% to 90% correct.
Using the same sounds, human performance was compared to a classification
model that used only spectral information. The classification model
compared each test sound to a set of prototypical spectra for both the oboe
and saxophone. To derive prototypical instruments, Brown (1999) used
oboe and saxophone passages (about 1 min in length) to derive an overall
spectral envelope averaged across notes. The overall envelope portrays the
sets of overlapping sound body resonances that amplify the excitation in
different frequency ranges. These overlapping resonances are termed formants,
and a formant in the 800–1000 Hz range would amplify the fourth
harmonic of 200–250 Hz excitations, the third harmonic of 300–333 Hz
excitations, and the second harmonic of 400–500 Hz excitations and
800–1000 Hz fundamental excitations. Thus, across a set of notes, there
would be consistent energy in the 800–1000 Hz range and that formant
would be part of the signature of a particular instrument.
To measure the performance of the computer model, the short segments
presented to the listeners were partitioned into 23 ms windows, and the
spectral envelopes of the windows were compared to the prototypical envelopes.
The computer model then calculated the likelihood that the segment
was played by the oboe or by the saxophone. Overall, the computer
was better than human listeners at identifying the oboe but equal at identifying
the saxophone. The success of the computer model demonstrates that
the spectra information was sufficient to distinguish between the two instruments,
but of course it does not unambiguously demonstrate that human
listeners are making use of the same information.
In subsequent work, J. C. Brown et al. (2001) used four wind instruments,
oboe, saxophone, clarinet, and flute. The results were similar to the
previous work: human and computer identification were roughly identical.
For the computer model, the most effective spectral information was the
shape of the spectrum due to overlapping resonances or the jaggedness of the
spectrum measured by the variation in the amplitudes of the harmonics. A detailed
spectral envelope description yielded better performance than just the
frequency of the spectral centroid, which is based on only one number.
Two studies illustrate that the spectral envelope can be used to discriminate
among natural events. Freed (1990) investigated whether listeners
could judge the hardness of mallets when they struck aluminum cooking
pots. The mallets were made of metal, wood, rubber, cloth-covered wood,
felt, and felt-covered rubber. The judges based their ratings on the spectra.
Mallets were judged as hard if the sound energy initially was concentrated
at higher frequencies and then shifted to lower frequencies over time (about
300 ms). As can be imagined easily, the harder mallets created a louder
sound, which also affected the hardness judgments. X. Li, Logan, and