Proceedings Fonetik 2009 - Institutionen för lingvistik

More documents

Recommendations

Info

Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm UniversityFactors affecting visual influence on heard vowel roundedness:Web experiments with Swedes and TurksHartmut TraunmüllerDepartment of Linguistics, University of StockholmAbstractThe influence of various general and stimulusspecificfactors on the contribution of vision toheard roundedness was investigated by meansof web experiments conducted in Swedish. Theoriginal utterances consisted of the syllables/ɡyːɡ/ and /ɡeːɡ/ of a male and a female speaker.They were synchronized with each other inall combinations, resulting in four stimuli thatwere incongruent in vowel quality, two of themadditionally in speaker sex. One of the experimentswas also conducted in Turkish, using thesame stimuli. The results showed that visiblepresence of lip rounding has a weaker effect onaudition than its absence, except for conditionsthat evoke increased attention, such as when aforeign language is involved. The results suggestthat female listeners are more susceptibleto vision under such conditions. There was nosignificant effect of age and of discomfort feltby being exposed to dubbed speech. A discrepancyin speaker sex did not lead to reducedinfluence of vision. The results also showed thathabituation to dubbed speech has no deterioratingeffect on normal auditory-visual integrationin the case of roundedness.IntroductionIn auditory speech perception, the perceptualweight of the information conveyed by the visibleface of a speaker can be expected to varywith many factors.1) The particular phonetic feature and system2) Language familiarity4) The individual speaker and speech style3) The individual perceiver5) Visibility of the face / audibility of the voice6) The perceiver’s knowledge about the stimuli7) Context8) Cultural factorsMost studies within this field have been concernedwith the perception of place of articulationin consonants, like McGurk and MacDonald(1976). These studies have shown that thepresence/absence of labial closure tends to beperceived by vision. As for vowels, it is knownthat under ideal audibility and visibility conditions,roundedness is largely heard by vision,while heard openness (vowel height) is hardlyat all influenced by vision (Traunmüller &Öhrström, 2007). These observations make itclear that the presence/absence of features tendsto be perceived by vision if their auditory cuesare subtle while their visual cues are prominent.Differences between phonetic systems are alsorelevant. When, e.g. an auditory [ɡ] is presentedin synchrony with a visual [b], this is likely tofuse into a [ɡ͡b] only for perceivers who arecompetent in a language with a [ɡ͡b]. Others aremore likely to perceive a [ɡ] or a consonantcluster. The observed lower visual influence inspeakers of Japanese as compared with English(Sekiyama and Burnham, 2008) represents amore subtle case, whose cause may lie outsidethe phonetic system.The influence of vision is increased whenthe perceived speech sounds foreign (Sekiyamaand Tohkura, 1993; Hayashi and Sekiyama,1998; Chen and Hazan, 2007). This is referredto as the “foreign-language effect”.The influence of vision varies substantiallybetween speakers and speaking styles (Munhallet al., 1996; Traunmüller and Öhrström, 2007).The influence of vision also varies greatlybetween perceivers. There is variation with age.Pre-school children are less sensitive (Sekiyamaand Burnham, 2004) although even prelinguisticchildren show influence of vision(Burnham and Dodd, 2004). There is also asubtle sex difference: Women tend to be moresusceptible to vision (Irwin et al., 2006;Traunmüller and Öhrström, 2007).The influence of vision increases with decreasingaudibility of the voice, e.g. due tonoise, and decreases with decreasing visibilityof the face, but only very litte with increasingdistance up to 10 m (Jordan, 2000).Auditory-visual integration works evenwhen there is a discrepancy in sex between avoice and a synchronized face (Green et al.,1991) and it is also robust with respect to whatthe perceiver is told about the stimuli. A minoreffect on vowel perception has, nevertheless,been observed when subjects were told the sex166
Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm Universityrepresented by an androgynous voice (Johnson,Strand and d’Imperio, 1991).Auditory-visual integration is robust to semanticfactors (Sams et al, 1998) but it is affectedby context, e.g. the vocalic context ofconsonants (Shigeno, 2002). It can also be affectedby the experimental method (e.g.,blocked vs. random stimulus presentation).It has been suggested that cultural conventions,such as socially prescribed gaze avoidance,may affect the influence of vision (Sekiyamaand Tohkura, 1993; Sekiyama, 1997).Exposure to dubbed films is another culturalfactor that can be suspected to affect the influenceof vision. The dubbing of foreign moviesis a widespread practice that often affects nearlyall speakers of certain languages. Since indubbed speech, the sound is largely incongruentwith the image, habituation requires learning todisrupt the normal process of auditory-visualintegration. Considering also that persons whoare not habituated often complain about discomfortand mental pain when occasionally exposedto dubbed speech, it deserves to be investigatedwhether the practice of dubbing deterioratesauditory-visual integration more permanentlyin the exposed populations.The present series of web experiments hadthe primary aim of investigating (1) the effectsof the perceiver’s knowledge about the stimuliand (2) those of a discrepancy between face andvoice (male/female) on the heard presence orabsence of liprounding in front vowels.Additional factors considered, without beingexperimentally balanced, were (3) sex and(4) age of the perceiver, (5) discomfort feltfrom dubbed speech, (6) noticed/unnoticedphonetic incongruence and (7) listening vialoudspeaker or headphones.The experiments were conducted in Swedish,but one experiment was also conducted inTurkish. The language factor that may discloseitself in this way has to be interpreted with caution,since (8) the “foreign-language effect” remainsconfounded with (9) effects due to thedifference between the phonetic systems.Most Turks are habituated to dubbedspeech, since dubbing foreign movies into Turkishis fairly common. Some are not habituated,since such dubbing is not pervasive. This allowsinvestigating (10) the effect of habituationto dubbed speech. Since dubbing into Swedishis only rarely practiced - with performances intendedfor children - adult Swedes are rarelyhabituated to dubbed speech.MethodSpeakersThe speakers were two native Swedes, a maledoctoral student, 29 years (index ♂ ), and a femalestudent, 21 years (index ♀ ). These weretwo of the four speakers who served for the experimentsreported in Traunmüller andÖhrström (2007). For the present experiment, aselection of audiovisual stimuli from this experimentwas reused.Speech materialThe original utterances consisted of the Swedishnonsense syllables /ɡyːɡ/ and /ɡeːɡ/. Eachauditory /ɡyːɡ/ was synchronized with eachvisual /ɡeːɡ/ and vice-versa. This resulted in 2times 4 stimuli that were incongruent in vowelquality, half of them being, in addition, incongruentin speaker (male vs. female).ExperimentsFour experiments were conducted with instructionsin Swedish. The last one of these was alsotranslated and conducted in Turkish, using thesame stimuli. The number of stimuli was limitedto 5 or 6 in order to facilitate the recruitmentof subjects.Experiment 1Sequence of stimuli (in each case first vowel byvoice, second vowel by face):e ♂ e ♂ , e ♀ y ♂ x, y ♂ e ♀ x, e ♂ y ♂ n, y ♀ e ♀ nFor each of the five stimuli, the subjects wereasked for the vowel quality they heard.“x” indicates that the subjects were also askedfor the sex of the speaker.“n” indicates that the subjects were also askedwhether the stimulus was natural or dubbed.Experiment 2In this experiment, there were two congruentstimuli in the beginning. After these, the subjectswere informed that they would next be exposedto two stimuli obtained by cross-dubbingthese. The incongruent stimuli and their orderof presentation were the same as in Exp. 1. Sequenceof stimuli:e ♀ e ♀ , y ♂ y ♂ , e ♀ y ♂ , y ♂ e ♀ , e ♂ y ♂ n, y ♀ e ♀ n167
Page 1 and 2:
Department of LinguisticsProceeding
Page 3 and 4:
Proceedings, FONETIK 2009, Dept. of
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Proceedings, FOETIK 2009, Dept. of
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116: Proceedings, FONETIK 2009, Dept. of
Page 127 and 128: Proceedings, FOETIK 2009, Dept. of
Page 165: Proceedings, FONETIK 2009, Dept. of
Page 217 and 218:
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
Page 225 and 226:
Page 227:
Department of LinguisticsPhonetics
show all

Proceedings Fonetik 2009 - Institutionen för lingvistik

Create successful ePaper yourself

Delete template?

Save as template?