1 - LumenVox

Tuning Grammars
There are many places to make effective changes, but generally, we have
found grammars to be the easiest and most effective place to start. In
this segment, we will look at how to detect errors and modify grammars
to optimize performance.
Grammar Terms
In-Grammar (IG) and Out-of-Grammar (OOG) are labels that look at whether the ASR matches
a path in the grammar with what the caller actually said. If it can, then the spoken words are
In-Grammar, if not, the spoken words are considered Out-of-Grammar.
Confidence scores indicate the ASR's certainty about the answer it returns.
Confirmations are dialog techniques to help the speech application avoid making a mistake, in
cases where the results are ambiguous.
Substitutions are a particular kind of error the ASR makes; this occurs when the result from the
ASR does not match the words the caller said.
LumenVox’s development and support
staff has been very responsive to our
requirements and issues.
Out-Of-Grammar Indicators
There are a number of ways to determine whether or not the error is due to an Out-of-Grammar
issue. The easiest, most efficient way to discover this is to use the tools provided by the platform
or ASR provider. Pre-configured reports will usually highlight these issues up front.
If Out-of-Grammar is a big problem, you will likely receive many customer complaints and low
completion and usage rates. Call lots will also have many "No Matches" or empty results. Finally,
when you do obtain results, the confidence score will be significantly lower than the rest of the
application.
As the call logs are transcribed, look for low accuracies. ASRs will usually get anything that is
In-Grammar, so if there is still low accuracy, look for a bunch of Out-of-Grammar speech.
The other good indicator is a high Out-of-Grammar rate. Generally, this is a direct indication that
callers are not saying things your grammar understands.
There are a few reasons for Out-of-Grammar problems, most of which are easy to resolve.
One common reason is that the grammar designer simply forgot to add items, but callers are asking
for them. Leaving "next" out of a navigation grammar, or forgetting to add a product name is not
uncommon.
Another common error is forgetting common synonyms, for example, 'copier', but not 'Xerox', or
different dialectal versions such as 'soda' in the west, and 'pop' in the South. Usually, you can just
add the missing items.
In-Grammar Indicators
Typical In-Grammar issues will often be oriented towards improving the confidence scores of
recognition, but you will also confront misrecognitions.
Doug Behl
President of Malibu Software
What kinds of issues arise frequently?
Regularly confused phrases are fairly common, and often result because two or more phrases sound
quite similar. Another common issue is the result of bad pronunciations in the grammar. ASRs
provide a methodology for arriving at pronunciations for words that aren't in the dictionary, but the
automatic pronunciations are not always the best.
So, how can we handle this?
For regularly confused phrases, differentiate them by choosing alternate ways to describe the
words. For bad pronunciations, you must add the pronunciations to the dictionary or grammar.
There are tools for helping with this task, although they are nearly always ASR-specific.
54 55