DAGA 2010 - Deutsche Gesellschaft für Akustik eV

68 DAGA 2010 Programm
Di. 15:45 Grashof C 20 Auditorisch-visuelle Sprache
Image-based Talking Head: Analysis and Synthesis
K. Liu und J. Ostermann
Leibniz Univ. Hannover, Institut für Informationsverarbeitung
The development of modern human-computer interfaces and their applications
such as E-Learning and web-based information services has
been the focus of the computer graphics community in recent years.
Image-based approaches for animating faces have achieved realistic
talking heads. In this paper, our image-based talking head system is
presented, which includes two parts: analysis and synthesis. In the analysis
part, a subject reading a predefined corpus is recorded first. The
recorded audio-visual data is analyzed in order to create a database
containing a large number of normalized mouth images and their related
information. The synthesis part generates natural looking talking heads
from phonetic transcripts by the unit selection algorithm. The phonetic
transcripts can be extracted from a TTS (Text-To-Speech) system (for
text-driven animation) or from speech by an aligner (for speech-driven
animation). The unit selection is to select and concatenate appropriate
mouth images from the database by minimizing two costs: lip synchronization
and smoothness. The lip synchronization measures how well the
unit fits to the phonetic context, and the smoothness cost measures how
well two units join together. Finally, the mouth images are stitched at the
correct position on the face of a recorded video sequence and the talking
head is displayed.
Di. 16:10 Grashof C 20 Auditorisch-visuelle Sprache
Adaptation of a Talking Head System to a Different Language
M. Zelezny und Z. Krnoul
University of West Bohemia, Plzen (CZ)
This paper presents new techniques that were developed in order to adapt
a talking head system for a different language. Originally, the system
is developed for the Czech language. Methods for generation and animation
of a 3D face model, speech data processing, and the whole system
training were designed for the same language. Recently, experiments
were carried out on adaptation of the system to the English and Dutch
languages. This paper generalizes these approaches for other languages.
Key steps of the methods that are needed to adhere and proposed
software tools to use are mentioned. The process of an audio-visual database
recording is summarized with the aim to get result data suitable
for the lip-tracking method used in the training system. Requirements for
a speaker, lighting and acoustical conditions, and data annotation requirements
are mentioned. The steps needed for the lip-tracking method
are a list of basic speech units for given language, segmentation of the
recorded data, and preparation of speech segments for settings of the
SAT (Selection of Articulatory Targets) method. At the end are mentioned
requirements for the TTS (text-to-speech) system so that from the