Photo-realistic text-driven Malay talking head with multiple expression

Abstract

In this decade of Information Technology, Talking Heads become more and more common as parts of modern computer-user interface. A talking head can add entertainment value to a program and make it more engaging. While Talking Heads with multiple expressions, it can let listeners to identify speaker's emotional state easily. But, generating animated talking heads that look like real people is a very challenging task, and so far all synthesized heads are still far from reaching this goal. Hence, the purpose of this work is to construct a natural, photo-realistic in appearance of Talking Heads which exhibit proper articulation by the lips that are synchronized with the speech. Moreover, realistic head movements, as well as emotional expressions that harmonize with the spoken text also added. And these Talking Heads are driven by English and Malay text. Photogram Metrics technique is applied to a model with photo-realistic face, while Sample-Based approach is implemented in facial movements of the Talking Heads to bypass the complexities linked to analytical modeling. Then, the Talking Heads will form the Audio-Visual Text-to-Speech Synthesis (AV-TTS) system and transformed into modules so that can be exported to any application.