Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

A Hybrid Approach for Facial Performance Analysis and Editing

: Paier, W.; Kettern, M.; Hilsmann, A.; Eisert, P.


IEEE transactions on circuits and systems for video technology 27 (2017), Nr.4, S.784-797
ISSN: 1051-8215
ISSN: 1558-2205
European Commission EC
FP7; 288369; RE@CT
Bundesministerium für Bildung und Forschung BMBF
03ZZ0407; 3DGIM
Fraunhofer HHI ()

As of today, fine-grained editing of facial performances in movie and video production requires either retouching every single frame or creating a highly detailed CGI model of the actor, both of which is restricted to high-budget productions. In this paper, we present an example-based approach for facial performance editing that achieves realistic results with standard equipment and very little manual intervention. Based on a model-free surface tracking approach, temporally consistent dynamic texture sequences are extracted from multiple video streams. Using such geometry-plus-texture sequences allows transferring facial expressions/performances between videos, which enables the editor, for example, to change the facial expression while leaving the rest of the video untouched. Moreover, concatenating and/or looping these sequences in a motion-graph-like manner offers a convenient way of composing novel facial performances from multiple source videos. Finally, we present a blending method to seamlessly concatenate texture/mesh sequences and to insert a composed facial performance in a target video.