Superadditivity in multisensory integration: putting the ... - CI Wiki
Superadditivity in multisensory integration: putting the ... - CI Wiki
Superadditivity in multisensory integration: putting the ... - CI Wiki
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
SUPERADDITIVITY IN MULTISENSORY INTEGRATION<br />
NEUROREPORT<br />
1.0<br />
Saccadic RT and <strong>in</strong>verse effectiveness<br />
12.0<br />
Manual RT and <strong>in</strong>verse effectiveness<br />
Normalized (re: visual) RT<br />
0.8<br />
0.6<br />
0.4<br />
0.2<br />
Vis<br />
Aud<br />
V + A<br />
Percentage of reduction <strong>in</strong> RT<br />
10.0<br />
8.0<br />
6.0<br />
4.0<br />
2.0<br />
VA low<br />
T low A low<br />
VA high<br />
VT low<br />
T high A high<br />
VT high<br />
0.0<br />
−21 dB −18 dB −12 dB −6 dB<br />
Auditory signal to noise ratio<br />
Fig. 4 Saccadic reaction time and <strong>in</strong>verse e¡ectiveness. This ¢gure,<br />
which was adapted from Figure 10 of Corneil et al. [7], illustrates <strong>the</strong><br />
relationship between stimulus strength and <strong>the</strong> decrements <strong>in</strong> saccadic<br />
reaction time (RT) result<strong>in</strong>g from comb<strong>in</strong><strong>in</strong>g visual and auditory stimuli.<br />
Human <strong>in</strong>dividuals were <strong>in</strong>structed to make saccades as quickly as possible<br />
to a visual stimulus, an auditory stimulus, or a spatially congruent<br />
visual^auditory stimulus. Auditory stimuli were presented at di¡erent<br />
signal strengths ( 21, 18, 12, and 6 dB) with respect to background<br />
noise. Plotted are mean RTs for saccades to <strong>the</strong> auditory stimulus alone<br />
(white bars) and <strong>the</strong> auditory^visual stimulus pair (gray bars) as a function<br />
of <strong>in</strong>creas<strong>in</strong>g auditory signal strength. All mean RTs are normalized relative<br />
to that for saccades to <strong>the</strong> visual stimulus alone (black bars ^ value of<br />
1.0). As expected, mean RT for <strong>the</strong> auditory stimulus (white bars) decl<strong>in</strong>es<br />
monotonically as a function of <strong>in</strong>creas<strong>in</strong>g auditory stimulus strength<br />
(<strong>in</strong>creas<strong>in</strong>g S/N ratio from left to right). Note, however, that S/N noise<br />
ratio also modulates <strong>the</strong> <strong>in</strong>£uence of <strong>multisensory</strong> <strong>in</strong>tegration such<br />
that mean RT for <strong>the</strong> cross-modal pair (gray bars) is shorter than that<br />
for <strong>the</strong> auditory stimulus alone (white bars) for <strong>the</strong> weakest auditory<br />
stimulus ( 21dB), but not for <strong>the</strong> most <strong>in</strong>tense auditory stimulus<br />
( 6 dB), a ¢nd<strong>in</strong>g consistent with <strong>in</strong>verse e¡ectiveness. It is also important<br />
to note that <strong>the</strong> apparent RT bene¢ts associated with cross-modal<br />
stimuli were not exclusive to <strong>the</strong> lowest S/N ratio, but appear to be<br />
present and of <strong>in</strong>termediate magnitude for <strong>the</strong> <strong>in</strong>termediate S/N ratios<br />
( 12 and 18 dB).<br />
(eyes, head, ears, body) to <strong>the</strong>se stimuli (see [28,29] for<br />
reviews). It is reasonable to presume that any stimulusrelated<br />
factor that leads to an augmented neural response <strong>in</strong><br />
<strong>the</strong> superior colliculus also <strong>in</strong>creases <strong>the</strong> salience of that<br />
stimulus, and with it <strong>the</strong> probability that it will elicit an<br />
orient<strong>in</strong>g response. It is well established that highly salient<br />
stimuli (e.g. bright, loud) provoke behavioral responses<br />
more reliably and more rapidly. Fur<strong>the</strong>rmore, <strong>the</strong>se same<br />
stimuli evoke neural responses that are of shorter latency,<br />
more vigorous, and less variable whe<strong>the</strong>r considered from<br />
<strong>the</strong> perspective of a s<strong>in</strong>gle neuron or populations of<br />
neurons. In many <strong>in</strong>stances, it seems justified to <strong>in</strong>fer a<br />
causal relationship from such behavior/neural correlates<br />
(e.g. greater activity level, shorter reaction time).<br />
Consider<strong>in</strong>g superadditivity as a context-limited phenomenon<br />
with<strong>in</strong> <strong>the</strong> broader spectrum of <strong>multisensory</strong><br />
<strong>in</strong>teractions, it seems self-evident that <strong>multisensory</strong> behavioral<br />
phenomena, at least those mediated by <strong>the</strong> superior<br />
colliculus, are not wholly dependent on supral<strong>in</strong>ear <strong>in</strong>teractions<br />
between <strong>the</strong> senses. Whereas one might reasonably<br />
expect that such <strong>in</strong>teractions contribute to <strong>the</strong> most potent<br />
behavioral effects observed when very weak unisensory<br />
stimuli are comb<strong>in</strong>ed, it should be recognized that simple<br />
0.0<br />
V+A V+T T+A<br />
Fig. 5 Manual reaction times and <strong>in</strong>verse e¡ectiveness. Shown here are<br />
<strong>the</strong> decrements <strong>in</strong> manual reaction time (RT) result<strong>in</strong>g from <strong>the</strong> addition<br />
of a stimulus from a second modality (i.e. cross-modal versus modalityspeci¢c).<br />
They are <strong>in</strong>versely related to stimulus <strong>in</strong>tensity. The data are<br />
taken fromTable 4 of Diederich and Colonius [6]. In <strong>the</strong>ir experiment, human<br />
<strong>in</strong>dividuals were <strong>in</strong>structed to depress a response button with each<br />
hand upon detect<strong>in</strong>g any stimulus, and manual reaction times were measured.<br />
Stimuli consisted of a visual £ash, auditory pure tone, vibratory<br />
tactile stimulus, or some comb<strong>in</strong>ation of <strong>the</strong>se three stimuli. Plotted are<br />
<strong>the</strong> % reductions <strong>in</strong> RT for <strong>the</strong> visual^auditory, tactile^visual, and tactile^<br />
auditory stimulus comb<strong>in</strong>ations. For each pair<strong>in</strong>g, <strong>the</strong> percent reduction<br />
<strong>in</strong> mean RT is plotted with reference to that for <strong>the</strong> modality-speci¢c<br />
component stimulus that yielded <strong>the</strong> shortest mean RTaccord<strong>in</strong>g to <strong>the</strong><br />
general formula: % reduction <strong>in</strong> RT¼m<strong>in</strong>(RT 1 , RT 2 ) (RT 1 + 2 )/m<strong>in</strong>(RT 1 ,<br />
RT 2 ) 100, where RT 1 and RT 2 are mean RTs for responses to each of <strong>the</strong><br />
stimulus components and RT 1 + 2 is <strong>the</strong> mean RT for responses to <strong>the</strong><br />
stimulus comb<strong>in</strong>ation. Note that for each cross-modal stimulus, <strong>the</strong> higher<br />
<strong>in</strong>tensity comb<strong>in</strong>ation (gray bars) yielded a proportionally lower reduction<br />
<strong>in</strong> RT than did <strong>the</strong> lower <strong>in</strong>tensity comb<strong>in</strong>ation (black bars), thus<br />
illustrat<strong>in</strong>g that <strong>the</strong> concept of <strong>in</strong>verse e¡ectiveness applies to manual<br />
RT. Note also that all stimuli were suprathreshold, illustrat<strong>in</strong>g that <strong>the</strong><br />
behavioral bene¢ts for <strong>multisensory</strong> <strong>in</strong>tegration are not exclusive to near<br />
threshold stimulation and that behavioral bene¢ts, albeit attenuated, are<br />
also observed for higher <strong>in</strong>tensity comb<strong>in</strong>ations. Stimulus pair<strong>in</strong>gs VA low ,<br />
VA high , VT low , VT high , T low A low , T high A high correspond to VA 70 , VA 90 , T 1 V,<br />
T 3 V,T 1 A 70 ,T 3 A 90 , respectively.<br />
summation, and even subl<strong>in</strong>ear comb<strong>in</strong>ations, of <strong>in</strong>dependent<br />
<strong>in</strong>puts would be expected to have behavioral manifestations<br />
by virtue of yield<strong>in</strong>g substantial <strong>in</strong>creases <strong>in</strong> superior<br />
colliculus activity. The predicted <strong>in</strong>verse trend has been<br />
demonstrated empirically for behavior; <strong>the</strong> <strong>in</strong>fluence of<br />
comb<strong>in</strong><strong>in</strong>g stimuli on behavioral measures such as localization<br />
or reaction time does, <strong>in</strong> fact, decrease with <strong>in</strong>creas<strong>in</strong>g<br />
salience of <strong>the</strong> unisensory components, and this pr<strong>in</strong>ciple<br />
seems to apply equally well to orientation of gaze (e.g. [7];<br />
see Fig. 4) or limb movement (e.g. [6]; see Fig. 5). Note,<br />
however, that <strong>the</strong> examples depicted <strong>in</strong> <strong>the</strong>se illustrations<br />
suggest that <strong>the</strong> behavioral effects of <strong>multisensory</strong> <strong>in</strong>tegration,<br />
whereas certa<strong>in</strong>ly greatest for <strong>the</strong> weakest stimuli, are<br />
not exclusive to such comb<strong>in</strong>ations.<br />
The concept of <strong>in</strong>verse effectiveness as it applies to<br />
behavior is <strong>in</strong>tuitive and, as discussed above, an analog<br />
(and perhaps its neural correlate) is evident <strong>in</strong> <strong>the</strong> responses<br />
of superior colliculus <strong>multisensory</strong> neurons: comb<strong>in</strong>ations<br />
of very weakly effective modality-specific stimuli result <strong>in</strong><br />
proportionately larger enhancements than do comb<strong>in</strong>ations<br />
of highly effective stimuli (see Figs 1–3). Given that<br />
<strong>multisensory</strong> phenomena arise via convergence (and, <strong>in</strong><br />
Vol 18 No 8 28 May 2007 791<br />
Copyright © Lipp<strong>in</strong>cott Williams & Wilk<strong>in</strong>s. Unauthorized reproduction of this article is prohibited.