1 - LumenVox

Tuning Processes
LumenVox's Speech Tuner provides full support for LumenVox's Speech
Recognition Engine, Nuance 8.5, ScanSoft OSR 2, and other ASRs. The
Speech Tuner allows you to work with any supported ASR via a single
interface.
LumenVox is an active supporter of the Tools committee in the VXML
Forum, and is working to help define standard logging information, to
help ease the tuning process.
The tuning process involves three easy steps:
Import Data.
1
2
3
The basic process is simple. Users import call log data into
the Speech Tuner database. All information stored by the call
log is available in the Speech Tuner. In most cases, log fields
between ASR engines are very similar; when the information
differs, every effort is made to preserve the original data.
Each special case is fully documented.
Transcribe Speech.
Transcribers can type the text of the caller's speech directly
into the Speech Tuner. Once the audio is transcribed, the
Tuner compares audio transcripts with the speech engine
results to determine accuracy, greatly reducing errors
associated with hand evaluations. If semantic interpretations
are available, the transcriber can also mark whether the
semantic interpretation was correct or incorrect. The
transcripts are evaluated using the actual decode grammar,
producing measurements such as word-error-rate, in- and
out-of-grammar rates, and semantic error rates.
Test Immediately.
Selecting an interaction in the Call Log automatically loads
the associated audio and grammar into the Tester. The
grammar can be edited, speech engine parameters set, and
individual recognition tests generated. The Speech Tuner
natively supports industry standard SRGS grammars. Once
a set of possible changes is identified, users can batch test
audio to evaluate performance, using those changes.
The Speech Tuner assumes the user possesses licensed
versions of the relevant ASR, that the ASR platform is up and
running, and that the platform is able to accept connections.
LumenVox Speech Tuner Database
The Speech Tuner communicates with an open-source, freeware database called SQLite
(www.sqlite.org). The Speech Tuner manages call log importing, searching, and exporting⎯so
users can focus on the task of tuning, not log management. The database is contained in a single
file, is easy to back up and transport, and can be queried using SQL-92 (see the SQLite website for
full details) from a variety of exterior tools. Other speech engine vendors are free to convert their
native logs to ones the engine understands. The format, content, and semantics of the LumenVox
Speech Tuner database are published.
The database maintains all the information contained in the original call log. The Speech Tuner
includes not only the decode grammar and ASR results, but also the decode platform, parameter
settings, alternative results, prompt audio, and pre- and post-processed audio.
Depending on the platform logging capabilities, the database can provide more advanced
information, such as ASR result alignments within the audio; the list of phonemes used in the
decode; and word, utterance, and semantic interpretation confidence measurements.
In addition, the Tuner stores all transcripts and evaluations within the call log. As transcripts are
entered into the Speech Tuner, they are automatically evaluated against the decode grammar.
These transcripts, and any notes or additional information, are stored directly into the database.
Individual scores⎯such as word error rate, semantic error rate, and in- and out-of-grammar
measurements⎯are stored along with their alignments, as well as information about how the scores
were reached.
Users can generate a variety of reports from these results, including error rate by grammar or
dialog, confusion matrices, transcription progress, and confidence thresholds for confirmation or
rejection settings.
In the future, LumenVox's Speech Tuner will also support back-end database replacement, for use in
enterprise level systems, where multiple users will be analyzing the same data simultaneously.
Companies who use an ODBC-capable database can replace, with certain SQL changes, the diskbased
SQLite system with an enterprise system such as MS SQL Server 2000, MySQL, PostgreSQL,
and/or Oracle.
LumenVox has created speech
recognition products that are easy to
code with and GUI-based tools, such as
the new Speech Tuner that greatly
simplifies post-deployment
maintenance.
Vern Baker
President of enGenic
Corporation
30 31