Other pages of interest

The SYNFACE project

Department of Phonetics and Linguistics at UCL -Home department for the project

Synthesised talking face derived from speech for hearing disabled users of voice channels

Funded by CEC Framework V IST program (reference number IST-2001-33327)

October 2001 - December 2004.

SYNFACE project home page at KTH

This work is an EU-funded project in collaboration with:

 The project aimed to develop a real-time system that delivers phonetically informative lipreading cues derived from parameters extracted from the acoustic speech signal. The starting point was a non-real time prototype developed for the Swedish language. This comprises a computer-generated wire-frame head model whose movement is controlled by a look-up table linked to the output of a Hidden-Markov-Model phoneme recogniser.

During the course of the project the head model was been developed further to add movements for sounds that occur in English and Dutch but not in Swedish. A series of studies of lipreading of the face by hearing-impaired and normally hearing people guided this development and verified the usefulness of the head-model

A second major development has been to assemble a real-time speech recogniser that operates sufficiently accurately with telephone speech. Because the head has to keep up with running speech, the recogniser is unique in working not only fast but on very short time samples of speech.

The project ran trials of a real-time version of the complete system that ran on a PC connected to a normal telephone - trials with hearing-impaired users showed great interest, even though the speech recognition performance was shown to be in need of further improvement

UCL's contribution to the project was led by Dr Andrew Faulkner. Other UCL staff associated with the project include Professor Ruth Campbell, Dr Mark Huckvale, and Dr Valerie Hazan.


Kattis is one of the SYNFACE characters - let her introduce herself - HERE

And see a video clip demonstrating the face in use over the telephone  - HERE

Non-technical overview of the project

Recent publications from the UCL project team

From Eurospeech 2003 -  Karlsson I, Faulkner A, Salvi G (2003) SYNFACE - a talking face telephone. The Eurospeech Special Event on "Spoken Language Technology in E-inclusion", Proc of EuroSpeech 2003 [pdf]

From AVSP  2003 - Siciliano C, Faulkner A and Williams G (2003). Lipreadability of a Synthetic Talking Face in Normal Hearing and Hearing-Impaired Listeners. Proc International conference AVSP 2003, 205-208

Centre for Human Communication at UCL - New cross-departmental Research Centre

 

Talking Heads website -

Has many useful pages and links on facial animiation and speech readin