Proceedings Fonetik 2009 - Institutionen för lingvistik

Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm UniversityFactors affecting visual influence on heard vowel roundedness:Web experiments with Swedes and TurksHartmut TraunmüllerDepartment of Linguistics, University of StockholmAbstractThe influence of various general and stimulusspecificfactors on the contribution of vision toheard roundedness was investigated by meansof web experiments conducted in Swedish. Theoriginal utterances consisted of the syllables/ɡyːɡ/ and /ɡeːɡ/ of a male and a female speaker.They were synchronized with each other inall combinations, resulting in four stimuli thatwere incongruent in vowel quality, two of themadditionally in speaker sex. One of the experimentswas also conducted in Turkish, using thesame stimuli. The results showed that visiblepresence of lip rounding has a weaker effect onaudition than its absence, except for conditionsthat evoke increased attention, such as when aforeign language is involved. The results suggestthat female listeners are more susceptibleto vision under such conditions. There was nosignificant effect of age and of discomfort feltby being exposed to dubbed speech. A discrepancyin speaker sex did not lead to reducedinfluence of vision. The results also showed thathabituation to dubbed speech has no deterioratingeffect on normal auditory-visual integrationin the case of roundedness.IntroductionIn auditory speech perception, the perceptualweight of the information conveyed by the visibleface of a speaker can be expected to varywith many factors.1) The particular phonetic feature and system2) Language familiarity4) The individual speaker and speech style3) The individual perceiver5) Visibility of the face / audibility of the voice6) The perceiver’s knowledge about the stimuli7) Context8) Cultural factorsMost studies within this field have been concernedwith the perception of place of articulationin consonants, like McGurk and MacDonald(1976). These studies have shown that thepresence/absence of labial closure tends to beperceived by vision. As for vowels, it is knownthat under ideal audibility and visibility conditions,roundedness is largely heard by vision,while heard openness (vowel height) is hardlyat all influenced by vision (Traunmüller &Öhrström, 2007). These observations make itclear that the presence/absence of features tendsto be perceived by vision if their auditory cuesare subtle while their visual cues are prominent.Differences between phonetic systems are alsorelevant. When, e.g. an auditory [ɡ] is presentedin synchrony with a visual [b], this is likely tofuse into a [ɡ͡b] only for perceivers who arecompetent in a language with a [ɡ͡b]. Others aremore likely to perceive a [ɡ] or a consonantcluster. The observed lower visual influence inspeakers of Japanese as compared with English(Sekiyama and Burnham, 2008) represents amore subtle case, whose cause may lie outsidethe phonetic system.The influence of vision is increased whenthe perceived speech sounds foreign (Sekiyamaand Tohkura, 1993; Hayashi and Sekiyama,1998; Chen and Hazan, 2007). This is referredto as the “foreign-language effect”.The influence of vision varies substantiallybetween speakers and speaking styles (Munhallet al., 1996; Traunmüller and Öhrström, 2007).The influence of vision also varies greatlybetween perceivers. There is variation with age.Pre-school children are less sensitive (Sekiyamaand Burnham, 2004) although even prelinguisticchildren show influence of vision(Burnham and Dodd, 2004). There is also asubtle sex difference: Women tend to be moresusceptible to vision (Irwin et al., 2006;Traunmüller and Öhrström, 2007).The influence of vision increases with decreasingaudibility of the voice, e.g. due tonoise, and decreases with decreasing visibilityof the face, but only very litte with increasingdistance up to 10 m (Jordan, 2000).Auditory-visual integration works evenwhen there is a discrepancy in sex between avoice and a synchronized face (Green et al.,1991) and it is also robust with respect to whatthe perceiver is told about the stimuli. A minoreffect on vowel perception has, nevertheless,been observed when subjects were told the sex166