MPEG-4 facial animation in video analysis and synthesis

MPEG-4 supports the de.nition, encoding, transmission, and animation of 3-D head and body models. These features can be used for a variety of di.erent applications ranging from low bit-rate video coding to character and avatar animation. In this paper, an entire system for the analysis of facial expressions from image sequences and their synthesis is presented. New methods for the estimation of MPEG-4 facial animation parameters as well as scene illumination are proposed. Experiments for di.erent applications demonstrate the potential of using facial animation techniques in video analysis and synthesis. A model-based codec is presented that is able to encode head-and-shoulder video sequences at bit-rates of about 1 kbit/s. Besides the low bit-rate, many enhancements and scene modi.cations can be easily applied, like scene lighting changes or cloning of expressions for character animation. But also for the encoding of arbitrary sequences, 3-D knowledge can help to increase the coding e.ciency. With our model-aided codec, bit-rate reductions of up to 45 % at the same quality can be achieved in comparison to standard hybrid video codecs.