Proceedings Fonetik 2009 - Institutionen för lingvistik

More documents

Recommendations

Info

Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm UniversityThe animations with synthetic tonguemovements (AVS) were created using a rulebasedvisual speech synthesizer developed forthe face (Beskow, 1995). For each viseme, targetvalues may be given for each parameter(i.e., articulatory feature). If a certain feature isunimportant for a certain phoneme, the target isleft undecided, to allow for coarticulation.Movements are then created based on the specifiedtargets, using linear interpolation andsmoothing. This signifies that a parameter thathas not been given a target for a phoneme willmove from and towards the targets in the adjacentphonemes. This simple coarticulationmodel has been shown to be adequate for facialmovements, since the synthesized face gesturessupport speech perception (e.g., Agelfors et al.1998; Siciliano et al., 2003). However, it is notcertain that the coarticulation model is sufficientto create realistic movements for thetongue, since they are more rapid and more directlyaffected by coarticulation processes.StimuliThe stimuli consisted of short (3-6 words long)simple Swedish sentences, with an “everydaycontent”, e.g., “Flickan hjälpte till i köket” (Thegirl helped in the kitchen). The acoustic signalhad been recorded together with the Qualisys-Movetrack measurements, and was presentedtime-synchronized with the animations.In the perception test, 50 sentences werepresented to the subjects: 10 in acoustic only(AO) condition (Set S1), and 20 each in AVRand AVS condition (Sets S2 and S3). All sentenceswere acoustically degraded using anoise-excited three-channel vocoder (Siciliano,2003) that reduces the spectral details and createsa speech signal that is amplitude modulatedand bandpass filtered. The signal consistsof multiple contiguous channels of white noiseover a specified frequency range.In the classification test, 72 sentences wereused, distributed evenly over the four conditionsAVR or AVS with normal audio (AVRn,AVSn) and AVR or AVS with vocoded audio(AVRv, AVSv), i.e. 18 sentences per condition.SubjectsThe perception test was run with 30 subjects,divided into three groups I, II and III. The onlydifference between groups I and II was thatthey saw the audiovisual stimuli in oppositeconditions (i.e., group I saw S2 in AVS and S3in AVR; group II S2 in AVR and S3 in AVS).Group III was a control group that was presentedall sets in AO.The classification test was run with 22 subjects,11 of whom had previously participatedin the perception test. The subjects were dividedinto two groups I and II, again with theonly difference being that they saw each sentencein opposite condition (AVR or AVS).All subjects were normal-hearing, nativeSwedes, aged 17 to 67 years old (PT) and 12 to81 years (CT). 18 male and 12 female subjectsparticipated in the perception test and 11 ofeach sex in the classification test.Experimental set-upBefore the perception test, the subjects werepresented a short familiarization session, consistingof five VCV words and five sentencespresented four times, once each with AVRv,AVRn, AVSv and AVSn. The subjects in theperception test were unaware of the fact thatthere were two different types of animations.The stimuli order was semi-random (PT) orrandom (CT), but the same for all groups,which means that the relative AVR-AVS conditionorder was reversed between groups I andII. The order was semi-random (i.e., the threedifferent conditions were evenly distributed) inthe perception test to avoid that learning effectsaffected the results.Each stimulus was presented three times inthe perception test and once in the classificationtest. For the latter, the subjects could repeat theanimation once. The subjects then gave theiranswer by typing in the perceived words in theperception test, and by pressing either of twobuttons (“Real” or “Synthetic”) in the classificationtest. After the classification test, but beforethey were given their classification score,the subjects typed in a short explanation onhow they had decided if an animation was fromreal movements or not.The perception test lasted 30 minutes andthe classification test 10 minutes.Data analysisThe word accuracy rate was counted manuallyin the perception test, disregarding spelling andword alignment errors. To assure that the differentgroups were matched and remove differencesthat were due to subjects rather than conditions,the results of the control group on setsS2 and S3 were weighted, using a scale factordetermined on set S1 by adjusting the average32
Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm Universityof group III so that the recognition score on thisset was the same as for the group I+II.For the classification test, two measures μand ∆ were calculated for all subjects. The classificationscore μ is the average proportion ofcorrectly classified animations c out of N presentations.The discrimination score ∆ insteadmeasures the proportion of correctly separatedanimations by disregarding if the label was corrector not. The measures are in the range0≤μ≤1 and 0.5≤∆≤1, with {μ, ∆}=0.5 signifyinganswers at chance level. The discriminationscore was calculated since we want to investigatenot only if the subjects can tell whichmovements are real but also if they can see differencesbetween the two animation types. Forexample, if subjects A and B had 60 and 12correct answers, μ=(60+12)/72=50% but∆=(36+24)/72=67%, indicating that consideredas a group, subject A and B could see the differencebetween the two types of animations,but not tell which were which.The μ, ∆ scores were also analyzed to findpotential differences due to the accompanyingacoustic signal, and correlations between classificationand word recognition score for thesubjects who participated in both tests.Word recognition score (%)70605040302010** ↕↕ *0Figure 2. Percentage of words correctly recognizedwhen presented in the different conditions AudioOnly (AO), Audiovisual with Real (AVR) or Syntheticmovements (AVS). The level of significancefor differences is indicated by * (p
Page 1 and 2: Department of LinguisticsProceeding
Page 3 and 4: Proceedings, FONETIK 2009, Dept. of
Page 31: Proceedings, FONETIK 2009, Dept. of
Page 41 and 42: Proceedings, FOETIK 2009, Dept. of
Page 83 and 84:
Proceedings, FONETIK 2009, Dept. of
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Proceedings, FOETIK 2009, Dept. of
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135 and 136:
Page 137 and 138:
Page 139 and 140:
Page 141 and 142:
Page 143 and 144:
Page 145 and 146:
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
Page 193 and 194:
Page 195 and 196:
Page 197 and 198:
Page 199 and 200:
Page 201 and 202:
Page 203 and 204:
Page 205 and 206:
Page 207 and 208:
Page 209 and 210:
Page 211 and 212:
Page 213 and 214:
Page 215 and 216:
Page 217 and 218:
Page 219 and 220:
Page 221 and 222:
Page 223 and 224:
Page 225 and 226:
Page 227:
Department of LinguisticsPhonetics
show all

Proceedings Fonetik 2009 - Institutionen för lingvistik

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?