Proceedings Fonetik 2009 - Institutionen för lingvistik

More documents

Recommendations

Info

Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm Universitylow amplitude oscillations are lost and the sequencesare interpreted as plateaus that arelonger (and therefore fewer within the analysiswindow) than those measured in speech segmentsproduced with higher F0. Such momentarychanges in the structure of the plateaus areinterpreted by the program's arbitrary code asindicating "deception". Under typical circumstances,flagging "lie" in association with loweringof F0 will give the illusion that the programis doing something sensible because F0tends to be lower when a speaker produces fillersduring hesitations than when the speaker'sspeech flows normally. Since the "lie-detector"is probably calibrated with responses to questionsabout obvious things the speaker will tendto answer using a typical F0 range that willgenerally be higher than when the speaker hasto answer to questions under split-attentionloads. Of course, when asked about events thatdemand recalling information, the speaker willtend to produce fillers or speak at a lowerspeech rate, thereby increasing the probabilityof being flagged by the system as attempting to"lie", although in fact hesitations or lowering ofF0 are known to be no reliable signs of deception.Intentionally or by accident, the illusion ofseriousness is further enhanced by the randomcharacter of the LVA outputs. This is a directconsequence of the technology's responses toboth the speech signal and all sorts of spuriousacoustic and digitalization accidents. The instabilityis likely to confuse both the speaker andthe "certified examiner", conveying the impressionthat the system really is detecting somebrain activity that the speaker cannot control 3and may not even be aware of! It may even givethe illusion of robustness as the performance isequally bad in all environments.Figure 2. The figures above show synthetic vowelsconstructed by algebraic addition of delayedversions of a natural glottal pulse. Thesewaveforms lead generate different “emotionaloutputs” depending on the relationship betweenthe F0 of the waveform being tested and the F0of the “calibration” waveform.low, the final portions of the vocal tract'sdamped responses to more sparse glottal pulseswill tend to achieve lower amplitudes in betweenconsecutive pulses. Given the technology'svery crude amplitude quantization, theseThe UK's DWP's evaluation of LVAThe UK's Department of Work and Pensionshas recently published statistics on the resultsof a large and systematic evaluation of theLVA-technology iii assessing 2785 subjects andcosting £2.4 million iv . The results indicate thatthe areas under the ROC curves for seven districtsvary from 0.51 to 0.73. The best of these3 Ironically this is true because the output is determinedby random factors associated with room acoustics, backgroundnoise, digitalization problems, distortion, etc.224
Proceedings, FONETIK 2009, Dept. of Linguistics, Stockholm Universityresults corresponds to a d' of about 0.9, which isa rather poor performance. But the numbers reportedin the table reflect probably the judgementsof the "Nemesysco-certified" personal 4in which case the meaningless results generatedby the LVA-technology may have been overriddenby personal's uncontrolled "interpretations"of the direct outcomes after listening torecordings of the interviews.Tabell 1. Evaluation results published by the UK'sDWP.NLow risk caseswith no change inbenefitHigh risk caseswith no change inbenefitLow risk caseswith change inbenefitHigh risk caseswith change inbenefitTrue NegativeFalse PositiveFalse NegativeTrue PositiveAUC of ROCCurveJobcentre Plus 787 354 182 145 106 0.54Birmingham 145 60 49 3 33 0.73Derwentside 316 271 22 11 12 0.72Edinburgh 82 60 8 8 6 0.66Harrow 268 193 15 53 7 0.52Lambeth 1101 811 108 153 29 0.52Wealden 86 70 7 8 1 0.51Overall 2785 1819 391 381 194 0.65ConclusionsThe essential problem of this LVA-technologyis that it does not extract relevant informationfrom the speech signal. It lacks validity.Strictly, the only procedure that might makesense is the calibration phase, where variablesare initialized with values derived from the fourvariables above. This is formally correct butrather meaningless because the waveformmeasurements lack validity and their reliabilityis low because of the huge information loss inthe representation of the speech signal used bythe LVA-technology. The association of ad hocwaveform measurements with the speaker’semotional state is extremely naive and ungroundedwishful thinking that makes thewhole calibration procedure simply void.4 An inquiry on the methodological details of the evaluationwas sent to the DWP on the 23 April 2009 but themethodological information has not yet been provided.In terms of “lie-detection”, the algorithm reliesstrongly on the variables associated withthe plateaus. Given the phonetic structure of thespeech signals, this predicts that, in principle,lowering the fundamental frequency and changingthe phonation mode towards a more creakyvoice type will tend to count as an indication oflie, in relation to a calibration made under modalphonation. Of course this does not haveanything to do with lying. It is just the consequencea common phonetic change in speakingstyle, in association with the arbitrary constructionof the “lie”-variable that happens to givemore weight to plateaus, which in turn are associatedwith the lower waveform amplitudestowards the end of the glottal periods in particularwhen the fundamental frequency is low.The overall conclusion from this study isthat from the perspectives of acoustic phoneticsand speech signal processing, the LVAtechnologystands out as a crude and absurdprocessing technique. Not only it lacks a theoreticalmodel linking its measurements of thewaveform with the speaker’s emotional statusbut the measurements themselves are so imprecisethat they cannot possibly convey useful information.And it will not make any differenceif Nemesysco “updates” in its LVA-technology.The problem is in the concept’s lack of validity.Without validity, “success stories” of “percentdetection rates” are simply void. Indeed, these“hit-rates” will not even be statistically significantdifferent from associated “false-alarms”,given the method’s lack of validity. Until proofof the contrary, the LVA-technology should besimply regarded as a hoax and should not beused for any serious purposes (Eriksson &Lacerda, 2007).ReferencesEriksson, A. and Lacerda, F. (2007). Charlatanryin forensic speech science: A problemto be taken seriously. Int Journal of Speech,Language and the Law, 14, 169-193.Liberman, A. (10-28-2003). Layered VoiceAnalysis (LVA). [6,638,217 B1]. US patent.i http://www.nemesysco.com/technology.htmlii http://www.nemesysco.com/technology-lvavoiceanalysis.htmliii http://spreadsheets.google.com/ccc?key=phNtm3LmDZEME67-nBnsRMwiv http://www.guardian.co.uk/news/datablog/2009/mar/19/dwp-voicerisk-analysis-statistics225
Page 1 and 2:
Department of LinguisticsProceeding
Page 3 and 4:
Proceedings, FONETIK 2009, Dept. of
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Proceedings, FOETIK 2009, Dept. of
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135 and 136:
Page 137 and 138:
Page 139 and 140:
Page 141 and 142:
Page 143 and 144:
Page 145 and 146:
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174: Proceedings, FONETIK 2009, Dept. of
Page 203 and 204: Proceedings, FOETIK 2009, Dept. of
Page 205 and 206: Proceedings, FOETIK 2009, Dept. of
Page 223: Proceedings, FONETIK 2009, Dept. of
Page 227: Department of LinguisticsPhonetics
show all

Proceedings Fonetik 2009 - Institutionen för lingvistik

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?