Superadditivity in multisensory integration: putting the ... - CI Wiki

SENSORYAND MOTOR SYSTEMS
NEUROREPORT
Superadditivity in multisensory integration:
putting the computation in context
Terrence R. Stanford and Barry E. Stein
Department of Neurobiology and Anatomy, Wake Forest University School of Medicine,Winston-Salem, North Carolina, USA
Correspondence to Terrence R. Stanford, Department of Neurobiology and Anatomy,Wake Forest University School of Medicine, Medical Center Blvd.,
Winston-Salem, NC 27157,USA
Tel: +1336 716 0359; fax: +1336 716 4534; e-mail: stanford@wfubmc.edu
Received 22 December 2006; accepted17 January 2007
Single-neuron studies have highlighted dramatic enhancements in
neural activity consequent to multisensory integration.Most notable
are‘superadditive’enhancements in which the multisensory response
exceeds the sum of those evoked by the modality-speci¢c
stimulus components individually. Although all multisensory
enhancements may have perceptual/behavioral consequences,
superadditivity, which suggests a nonlinear combination of modality-speci¢c
in£uences, seems to have had a disproportionate in£uence
within the multisensory literature. This in£uence has been
reinforced by the increasing application of noninvasive techniques
such as functional imaging and event-related potential recording,
which depend on response nonlinearities to demonstrate underlying
multisensory processes.In promoting the idea that many multisensory
behaviors may not rely on superadditivity, we consider
more recent single-neuron studies that place its incidence in
context. NeuroReport 18:787^792 c 2007 Lippincott Williams &
Wilkins.
Keywords: multisensory integration, single-neuron recording, superadditivity, superior colliculus
Introduction
During the past decade, the field of multisensory integration
has expanded dramatically and now encompasses most of
the emerging neuroscience methodologies, fosters both
clinical and technological applications, and challenges some
of the core assumptions about how sensory information is
sequestered and shared in the nervous system. As is often
the case for a rapidly expanding discipline, there is a
healthy tendency to reevaluate its fundamental tenets. For
multisensory integration, there is perhaps no more fundamental
principle than that of ‘multisensory enhancement’.
Multisensory enhancement, a term born out of the seminal
single-neuron electrophysiology experiments of the cat
superior colliculus, refers to the phenomenon that a neuron
receiving input from multiple sensory modalities responds
more vigorously to their simultaneous activation than to
activation of any single modality-specific channel (see Ref.
[1] for review). This intuitively obvious concept (e.g. two is
‘better’ than one) has inspired and continues to inspire all
manner of multisensory investigations. Furthermore, it has
provided a simple conceptual framework for interpreting
multisensory findings suggesting behavioral benefits associated
with signal enhancement in the central nervous
system, benefits that include improved stimulus detection
and more rapid and accurate orienting [2–8], and even the
improvement of visual hemi-neglect in brain-damaged
individuals [9].
To date, conceptual frameworks driven by the empirical
findings hinge on the idea that integration leads to a relative
increase in activity within the central nervous system. This
core assumption, which has great explanatory power for
considering behavioral benefits such as increased stimulus
detection and speeded reaction time, is strongly supported
by a wealth of neurophysiological data, including those
from single-neuron recording, event-related potential (ERP),
and functional imaging (fMRI) studies (see Refs. [10,11] for
recent reviews). It is perhaps not surprising, then, that
multisensory researchers have often placed particularly
heavy emphasis on evidence for the largest multisensory
enhancements. The single-neuron literature, for example, is
laden with examples in which neural responses to the
combination of stimuli from different modalities exceeds
(sometimes greatly) the sum of the responses to either
stimulus component presented individually (Fig. 1; see Ref.
[1] for review). So-called superadditive interactions are
interesting both from the perspective of the single neuron,
wherein they suggest nonlinear synaptic mechanisms at
work, and from the perspective of behavior, for which they
promise the greatest real-world benefits.
Whereas earlier single-neuron studies may have been
biased toward cases of superadditivity, studies using
noninvasive methods like fMRI and ERP, which have
become increasingly important tools in the study of multisensory
phenomena, are constrained to focus specifically on
response nonlinearities. In the case of multisensory enhancement,
these methodologies do not distinguish between
linear (i.e. additive) signal enhancements that are due
to recruitment of separate pools of unisensory neurons and
those that are the result of true multisensory convergence
and integration as, in either case, the response to a
good point.
the
resolution
can't tell
the two
apart.
0959-4965 c Lippincott Williams & Wilkins Vol 18 No 8 28 May 2007 78 7
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT
STANFORD AND STEIN
(a) (b) (c) (d)
V
V
20
A
A
15
+1207%
50
100 ms
x Impulses/trial
10
V only
A only
VA
5
0
V
A
VA
Fig.1 Superadditivity in multisensory integration: shown here in a ¢gure reproducing Figure 2 of Meredith and Stein [17] is an example of highly robust
multisensory enhancement and one that is clearly superadditive. In this experiment, the activity of a cat superior colliculus neuron was recorded in
response to a visual stimulus, an auditory stimulus, and their cross-modal combination. Left, top-to-bottom: the stimulus trace (in this case visual), rasters
of the impulses evoked during each stimulus presentation, a peristimulus time histogram of the summed impulses across trials, and a single oscilloscope
trace of the extracellular action potentials during a single trial.The second and third panels illustrate the same for the auditory (second panel) and
cross-modal (third panel) stimulus condition.Note that the vigor of the cross-modal response far exceeds that of either modality-speci¢c response.These
di¡erences are summarized in the bar graph (fourth panel, far right), which compares the mean number of impulses per trial for each stimulus condition
and quanti¢es the percent enhancement (1207) according to the index formula,VA max (V, A)/max (V, A) 100, whereVA is the response to the visualauditory
crossmodal stimulus and max (V, A) is the response to the most e¡ective modality-speci¢c component stimulus (visual in this example). Note
that the multisensory response of approximately18 impulses per trial far exceeds the sum of thevery weak responses to the individual visual and auditory
stimuli (V + A¼approx. two impulses/trial).
multisensory stimulus would approximate the sum of the
responses to its modality-specific components. For fMRI,
superadditivity, which is not readily explained by the
separate pool hypothesis, has become the litmus test for
identifying multisensory enhancements that are due to
integration by multisensory neurons (see Refs. [12,13] for
reviews). For ERP studies, superadditivity is also diagnostic
of multisensory integration, although its attribution to
enhancement requires certain additional assumptions
(see Refs. [14–16] discussion of these issues).
As enticing as it may be as a phenomenon, an unbalanced
emphasis on superadditivity poses a risk, a risk that is
certainly heightened by the increasing number of fMRI and
ERP studies, which, despite their considerable contributions,
are constrained by methodological limitations that
render multisensory integration synonymous with response
nonlinearity. A lack of due diligence on the part of author or
reader has the potential to distort our view of multisensory
representations within the brain and, as a consequence, our
understanding of the relationships between multisensory
integration and behavior. In this regard, it is especially
important to highlight results from the most recent singleneuron
studies that emphasize the wider spectrum of
multisensory interactions and, in doing so, place the
incidence (and, by inference, the importance) of superadditivity
in a broader context. Such studies provide
important constraints on models and conceptualizations of
how multisensory interactions within neural populations
are manifest in behavioral measures.
The prevalence of superadditivity in the superior
colliculus
Multisensory enhancement is the most fundamental manifestation
of integration at the single neuron level; however,
relatively early in their investigations of the superior
colliculus, Meredith and Stein [17] noted that, in relative
terms, the ‘greatest’ enhancements occurred for multisensory
combinations of the ‘weakest’ sensory stimuli (see
Fig. 2). Appropriately, this association was termed ‘inverse
78 8 Vol 18 No 8 28 May 2007
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

SUPERADDITIVITY IN MULTISENSORY INTEGRATION
NEUROREPORT
(a)
Optimal
V
A
A
V
40
30
100
200 ms
x Impulses/trial
20
+110%
10
(b)
V only A only VA
0
V A VA
Suboptimal
30
20
50
10
+258%
0
(c)
Minimal
30
20
50
10
+483%
Fig. 2 Inverse e¡ectiveness in multisensory integration: This ¢gure, adapted from Figure 8 of Meredith and Stein [17], demonstrates the principle of
inverse e¡ectiveness for a single neuron in the cat superior colliculus. Using the same format as Fig.1, the ¢gure illustrates that the magnitude of multisensory
enhancement depends on the e⁄cacy of the modality-speci¢c component stimuli, which were modulated by manipulating stimulus intensity.
According to the enhancement index formula of Fig. 1, which relates the multisensory response (VA) to that for the most e¡ective modality-speci¢c
stimulus (auditory in all cases here), multisensory enhancement grows from approximately 53% for the ‘optimal’ (a) condition to 50% for the ‘minimal’
(c) condition (values estimated from bar graphs at far right), thus demonstrating the principle of inverse e¡ectiveness as originally de¢ned. It is
also readily apparent from visual inspection of the bar graphs that the ‘optimal’ condition yields a multisensory response that is nominally ‘subadditive’
(VAoV + A), whereas the‘minimal’condition yields a response that is clearly ‘superadditive’ (VA44V + A).
0
effectiveness’ and, as noted below, it has a counterpart in
certain behavioral measures. With the principle of inverse
effectiveness, it was established early on that the magnitude
and probability of multisensory enhancement is strongly
influenced by the intensities, or more precisely, the efficacies
of the constituent stimuli. Despite the clear inference that
the likelihood of observing superadditivity must also then be
strongly stimulus dependent, evidence for inverse effectiveness
has not prevented a ‘superadditive’ ideal of the
multisensory interaction from seemingly gaining wide
acceptance. Ironically, awareness of inverse effectiveness
may have contributed to propagating this misconception by
permitting experimenters to tailor stimulus parameters to
reliably produce large and often superadditive multisensory
Vol 18 No 8 28 May 2007 78 9
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT
STANFORD AND STEIN
so
superadditivi
ty might not
be a stable
feature of a
neuron, but
dependent of
the
properties of
the stimuli.
enhancements. Doing so has been particularly valuable, if
not essential, for assessing how normal development
[18,19], altered sensory experience [20], or acute perturbations
of input (e.g. cortical inactivation) [21] affect the
incidence of multisensory integration within the superior
colliculus, but with the untoward effect of creating a biased
representation of superior colliculus multisensory interactions
within the literature.
In an effort to more fully characterize the nature of
multisensory integration for single neurons and populations
of neurons within the cat superior colliculus, recent studies
have eschewed the near-exclusive use of minimally effective
stimuli in favor of parametric manipulations of stimulus
intensity to modulate effectiveness over a broad range. In
one such study, Stanford et al. [22] quantitatively evaluated
the operation performed by superior colliculus multisensory
neurons for combinations of modality-specific stimuli
covering a wide range of efficacies. Consistent with the
many individual examples in the literature (e.g. see Figs 1
and 2c) this study verified that superadditivity is in fact
commonly observed when very ineffective modality-specific
stimuli are combined. They also, however, found that the
incidence of superadditive interactions fell precipitously
with increasing efficacy of the individual modality-specific
stimulus components. Indeed, across the broader range of
efficacies, the majority of the interactions in their sample
approximated linear summation of the modality-specific
inputs, with superadditive and subadditive interactions
defining the tails of a normal distribution (Fig. 3).
Analogous results were described by Perrault et al. [23],
who also evaluated multisensory interactions against a
benchmark of additivity with emphasis on relationships
between the multisensory operation and the dynamic
ranges of individual superior colliculus neurons. The results
were once again consistent with extant examples in the
literature, but also established the context in which such
previous examples should be considered. Perrault et al. [23]
demonstrated that weakly responsive neurons with very
compressed dynamic ranges (i.e. weakly responsive over a
broad range of stimulus intensities) were those most likely
to demonstrate superadditivity exclusively, whereas those
with more expansive and more linear dynamic ranges
tended to transition from superadditivity to additivity or
from additivity to subadditivity as stimulus intensity (and
efficacy) increased. These recent studies demonstrated that,
in the superior colliculus at least, superadditivity is but one
facet of multisensory integration, and one that is produced
under a very circumscribed range of circumstances, specifically
when the unisensory stimuli to be combined are
weakly effective. As one might have expected, the results of
these studies illustrate that the principle of ‘inverse
effectiveness’ can be extended beyond multisensory enhancement
to include the form of the multisensory
computation.
These larger surveys of superior colliculus integration are
particularly germane to consideration of an earlier study
by Populin and Yin [24], which stands out as the only
proponent of a contrarian and rather provocative conjecture
that superadditivity is an artifact of the anesthetic agent
used in most previous studies. This conclusion was based
on the fact that in their study of multisensory integration in
the superior colliculus of alert cats, Populin and Yin found
little evidence of superadditivity, instead reporting that
most multisensory interactions were either additive (linear)
Response mode proportion
100
80
60
40
20
0
0−2 2−4 4−6 6−8
Superadditive
Additive
Subadditive
8−10 10−12 12−14
Unisensory response sum (impulses/trial)
Fig. 3 The incidence of superadditivity declines rapidly with increasing
stimulus strength. Shown here, in a ¢gure adapted from Figure 6a of Stanford
et al. [22], is an illustration of how the neural integration of crossmodal
stimulus pairs depends on the strength of the modality-speci¢c
component stimuli. In this experiment, single neuron activity from the
cat superior colliculus was recorded in response to visual, auditory, and
visual-auditory stimuli across a wide range of stimulus intensities. In each
case, the magnitude of the response to the cross-modal stimulus was
evaluated with respect to a benchmark of simple summation of the responses
to the modality-speci¢c component stimuli (see Stanford et al.
[22] for details).The plot illustrates the relative likelihood of multisensory
responses that exceeded summation (superadditive¼¢lled circles), failed
to achieve summation (subadditive¼open squares), or were consistent
with summation (additive¼open circles). Note that the likelihood of observing
superadditivity fell dramatically as the e⁄cacy of the component
stimuli increased. Thus, superadditivity predominated only for very
weakly e¡ective auditory and visual stimulus components, stimuli for
which the predicted sum of the modality-speci¢c responses failed to
exceed 2^ 4 impulses/trial.
or subadditive (sublinear). Although the Populin and Yin
findings appeared to be an extreme departure from earlier
literature, including those from other studies in awake
animals (e.g. see Ref. [25]), their data overlap greatly with
the more recent studies showing that, in anesthetized cats,
additivity predominates for combinations of all but the
weakest modality-specific stimuli [22,23]. Populin and Yin
do not relate integration mode to stimulus efficacy for their
sample; however, the specific examples provided suggest
moderate efficacy and, at first glance, seem in line with the
most recent results from anesthetized cats. This, along with
both earlier [25] and more recent reports of superadditive
multisensory interactions in single neurons in a variety of
structures in awake, behaving animals (monkey cortex: [26];
rat thalamus: [27]), coupled with the data from fMRI and
ERP studies (see above) strongly suggest that a difference in
sampling and/or stimulus efficacy is the more parsimonious
explanation of the extreme paucity of superadditive
cases in their sample.
Implications for behavior
From a functional perspective, the issue of integrative
mechanism (i.e. linear or nonlinear) is relevant only to the
extent that it dictates the magnitude (and/or timing) of the
postsynaptic response. Neurons in the superior colliculus
represent salient visual, auditory, and tactile stimuli and
contribute to the formation of motor commands to orient
even if a
certain % of
trials with
superadd,
that would
manifest as
increased
activity in
study..
79 0 Vol 18 No 8 28 May 2007
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

SUPERADDITIVITY IN MULTISENSORY INTEGRATION
NEUROREPORT
1.0
Saccadic RT and inverse effectiveness
12.0
Manual RT and inverse effectiveness
Normalized (re: visual) RT
0.8
0.6
0.4
0.2
Vis
Aud
V + A
Percentage of reduction in RT
10.0
8.0
6.0
4.0
2.0
VA low
T low A low
VA high
VT low
T high A high
VT high
0.0
−21 dB −18 dB −12 dB −6 dB
Auditory signal to noise ratio
Fig. 4 Saccadic reaction time and inverse e¡ectiveness. This ¢gure,
which was adapted from Figure 10 of Corneil et al. [7], illustrates the
relationship between stimulus strength and the decrements in saccadic
reaction time (RT) resulting from combining visual and auditory stimuli.
Human individuals were instructed to make saccades as quickly as possible
to a visual stimulus, an auditory stimulus, or a spatially congruent
visual^auditory stimulus. Auditory stimuli were presented at di¡erent
signal strengths ( 21, 18, 12, and 6 dB) with respect to background
noise. Plotted are mean RTs for saccades to the auditory stimulus alone
(white bars) and the auditory^visual stimulus pair (gray bars) as a function
of increasing auditory signal strength. All mean RTs are normalized relative
to that for saccades to the visual stimulus alone (black bars ^ value of
1.0). As expected, mean RT for the auditory stimulus (white bars) declines
monotonically as a function of increasing auditory stimulus strength
(increasing S/N ratio from left to right). Note, however, that S/N noise
ratio also modulates the in£uence of multisensory integration such
that mean RT for the cross-modal pair (gray bars) is shorter than that
for the auditory stimulus alone (white bars) for the weakest auditory
stimulus ( 21dB), but not for the most intense auditory stimulus
( 6 dB), a ¢nding consistent with inverse e¡ectiveness. It is also important
to note that the apparent RT bene¢ts associated with cross-modal
stimuli were not exclusive to the lowest S/N ratio, but appear to be
present and of intermediate magnitude for the intermediate S/N ratios
( 12 and 18 dB).
(eyes, head, ears, body) to these stimuli (see [28,29] for
reviews). It is reasonable to presume that any stimulusrelated
factor that leads to an augmented neural response in
the superior colliculus also increases the salience of that
stimulus, and with it the probability that it will elicit an
orienting response. It is well established that highly salient
stimuli (e.g. bright, loud) provoke behavioral responses
more reliably and more rapidly. Furthermore, these same
stimuli evoke neural responses that are of shorter latency,
more vigorous, and less variable whether considered from
the perspective of a single neuron or populations of
neurons. In many instances, it seems justified to infer a
causal relationship from such behavior/neural correlates
(e.g. greater activity level, shorter reaction time).
Considering superadditivity as a context-limited phenomenon
within the broader spectrum of multisensory
interactions, it seems self-evident that multisensory behavioral
phenomena, at least those mediated by the superior
colliculus, are not wholly dependent on supralinear interactions
between the senses. Whereas one might reasonably
expect that such interactions contribute to the most potent
behavioral effects observed when very weak unisensory
stimuli are combined, it should be recognized that simple
0.0
V+A V+T T+A
Fig. 5 Manual reaction times and inverse e¡ectiveness. Shown here are
the decrements in manual reaction time (RT) resulting from the addition
of a stimulus from a second modality (i.e. cross-modal versus modalityspeci¢c).
They are inversely related to stimulus intensity. The data are
taken fromTable 4 of Diederich and Colonius [6]. In their experiment, human
individuals were instructed to depress a response button with each
hand upon detecting any stimulus, and manual reaction times were measured.
Stimuli consisted of a visual £ash, auditory pure tone, vibratory
tactile stimulus, or some combination of these three stimuli. Plotted are
the % reductions in RT for the visual^auditory, tactile^visual, and tactile^
auditory stimulus combinations. For each pairing, the percent reduction
in mean RT is plotted with reference to that for the modality-speci¢c
component stimulus that yielded the shortest mean RTaccording to the
general formula: % reduction in RT¼min(RT 1 , RT 2 ) (RT 1 + 2 )/min(RT 1 ,
RT 2 ) 100, where RT 1 and RT 2 are mean RTs for responses to each of the
stimulus components and RT 1 + 2 is the mean RT for responses to the
stimulus combination. Note that for each cross-modal stimulus, the higher
intensity combination (gray bars) yielded a proportionally lower reduction
in RT than did the lower intensity combination (black bars), thus
illustrating that the concept of inverse e¡ectiveness applies to manual
RT. Note also that all stimuli were suprathreshold, illustrating that the
behavioral bene¢ts for multisensory integration are not exclusive to near
threshold stimulation and that behavioral bene¢ts, albeit attenuated, are
also observed for higher intensity combinations. Stimulus pairings VA low ,
VA high , VT low , VT high , T low A low , T high A high correspond to VA 70 , VA 90 , T 1 V,
T 3 V,T 1 A 70 ,T 3 A 90 , respectively.
summation, and even sublinear combinations, of independent
inputs would be expected to have behavioral manifestations
by virtue of yielding substantial increases in superior
colliculus activity. The predicted inverse trend has been
demonstrated empirically for behavior; the influence of
combining stimuli on behavioral measures such as localization
or reaction time does, in fact, decrease with increasing
salience of the unisensory components, and this principle
seems to apply equally well to orientation of gaze (e.g. [7];
see Fig. 4) or limb movement (e.g. [6]; see Fig. 5). Note,
however, that the examples depicted in these illustrations
suggest that the behavioral effects of multisensory integration,
whereas certainly greatest for the weakest stimuli, are
not exclusive to such combinations.
The concept of inverse effectiveness as it applies to
behavior is intuitive and, as discussed above, an analog
(and perhaps its neural correlate) is evident in the responses
of superior colliculus multisensory neurons: combinations
of very weakly effective modality-specific stimuli result in
proportionately larger enhancements than do combinations
of highly effective stimuli (see Figs 1–3). Given that
multisensory phenomena arise via convergence (and, in
Vol 18 No 8 28 May 2007 791
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT
STANFORD AND STEIN
many cases, linear combination) of information on unisensory
channels, it is not surprising that the function
relating the behavioral products of multisensory integration
to stimulus intensity is qualitatively similar to that for
unisensory stimuli. For unisensory stimuli, the relationship
between stimulus intensity and stimulus detection or
reaction time is also one of diminishing returns; psychometric
functions of stimulus detection probability versus
stimulus intensity display the characteristic sigmoid shape,
positively accelerating near threshold and negatively accelerating
near maximal performance. Likewise, reaction time
decrements with stimulus intensity are described by an
inverse power function (Pieron’s Law) that prescribes large
reaction time decrements for near-threshold intensity increments
and little to no effect for increases in the highintensity
range [30,31].
The analogy drawn between increases in activity owing to
increments in within-modal stimulus intensity and those
due to the addition of stimuli from a second modality (i.e.
multisensory integration) is instructive; it reminds us that,
from the point of view of circuits that must ‘read out’
superior colliculus activity to produce behavior, the particular
mechanism that gave rise to an increase in activity is
irrelevant. With this in mind, we emphasize that superadditivity
is but one of several computations through which
multisensory integration enhances the neural representation
of sensory signals and the behaviors that depend on them.
Acknowledgement
Grant support: NS36916 and NS22543.
References
1. Stein BE, Meredith MA. The merging of the senses. Cambridge, MA: The
MIT Press; 1993.
2. Stein BE, Meredith MA, Huneycutt WS, McDade L. Behavioral indices
of multisensory integration: orientation to visual cues is affected by
auditory stimuli. J Cogn Neurosci 1989; 1:12–24.
3. Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R. Visual–auditory
interactions in sensorimotor processing: saccades versus manual
responses. J Exp Psychol Hum Percept Perform 1994; 20:131–153.
4. Nozawa G, Reuter-Lorenz PA, Hughes HC. Parallel and serial processes
in the human oculomotor system: bimodal integration and express
saccades. Biol Cybern 1994; 72:19–34.
5. Colonius H, Arndt P. A two-stage model for visual–auditory interaction
in saccadic latencies. Percept Psychophys 2001; 63:126–147.
6. Diederich A, Colonius H. Bimodal and trimodal multisensory enhancement:
effects of stimulus onset and intensity on reaction time. Percept
Psychophys 2004; 66:1388–1404.
7. Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ. Auditory–visual
interactions subserving goal-directed saccades in a complex scene.
J Neurophysiol 2002; 88:438-454.
8. Frens MA, Van Opstal AJ, Van der Willigen RF. Spatial and temporal
factors determine auditory–visual interactions in human saccadic eye
movements. Percept Psychophys 1995; 57:802–816.
9. Bolognini N, Rasi F, Coccia M, Ladavas E. Visual search improvement
in hemianopic patients after audio–visual stimulation. Brain 2005; 128:
2830–2842.
10. Calvert GA, Spence C, Stein BE. The handbook of multisensory processes.
Cambridge, MA: The MIT Press; 2004.
11. Foxe JJ, Schroeder CE. The case for feedforward multisensory
convergence during early cortical processing. NeuroReport 2005;
16:419–423.
12. Calvert GA. Crossmodal processing in the human brain: insights from
functional neuroimaging studies. Cereb Cortex 2001; 11:1110–1123.
13. Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE. On the use of
superadditivity as a metric for characterizing multisensory integration in
functional neuroimaging studies. Exp Brain Res 2005; 166:289–297.
14. Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE.
Multisensory auditory–somatosensory interactions in early cortical
processing revealed by high-density electrical mapping. Brain Res Cogn
Brain Res 2000; 10:77–83.
15. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ.
Multisensory auditory–visual interactions during early sensory
processing in humans: a high-density electrical mapping study. Brain
Res Cogn Brain Res 2002; 14:115–128.
16. Fort A, Giard M. In: Calvert GA, Spence C, Stein BE, editors. The handbook
of multisensory integration. Cambridge, MA: The MIT Press; 2004.
17. Meredith MA, Stein BE. Visual, auditory, and somatosensory
convergence on cells in superior colliculus results in multisensory
integration. J Neurophysiol 1986; 56:640–662.
18. Wallace MT, McHaffie JG, Stein BE. Visual response properties and
visuotopic representation in the newborn monkey superior colliculus.
J Neurophysiol 1997; 78:2732–2741.
19. Jiang W, Jiang H, Rowland BA, Stein BE. Multisensory orientation
behavior is disrupted by neonatal cortical ablation. J Neurophysiol 2006.
20. Wallace MT, Stein BE. Early experience determines how the senses will
interact. J Neurophysiol 2006.
21. Jiang W, Wallace MT, Jiang H, Vaughan JW, Stein BE. Two cortical areas
mediate multisensory integration in superior colliculus neurons.
J Neurophysiol 2001; 85:506–522.
22. Stanford TR, Quessy S, Stein BE. Evaluating the operations underlying
multisensory integration in the cat superior colliculus. J Neurosci 2005;
25:6499–6508.
23. Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT. Superior colliculus
neurons use distinct operational modes in the integration of multisenory
stimuli. J Neurophysiol 2005; 93:2575–2586.
24. Populin LC, Yin TC. Bimodal interactions in the superior colliculus of the
behaving cat. J Neurosci 2002; 22:2826–2834.
25. Wallace MT, Meredith MA, Stein BE. Multisensory integration in the
superior colliculus of the alert cat. J Neurophysiol 1998; 80:1006–1010.
26. Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI. Integration of
visual and auditory information by superior temporal sulcus neurons
responsive to the sight of actions. J Cogn Neurosci 2005; 17:377–391.
27. Komura Y, Tamura R, Uwano T, Nishijo H, Ono T. Auditory thalamus
integrates visual inputs into behavioral gains. Nat Neurosci 2005; 8:
1203–1209.
28. Sparks DL. Translation of sensory signals into commands for control of
saccadic eye movements: role of primate superior colliculus. Physiol Rev
1986; 66:118–171.
29. Hall WC, Moschovakis A. The superior colliculus: new approaches for
studying sensorimotor integration. Boca Raton, FL: CRC Press; 2004.
30. Pieron H. The sensations: their functions, processes and mechanisms. London:
Frederick Muller Ltd.; 1952.
31. Luce RD. Response times: their role in inferring elementary mental
organisation. New York: Clarendon Press; 1986.
79 2 Vol 18 No 8 28 May 2007
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.