Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

High detail flexible viewpoint facial video from monocular input using static geometric proxies

: Kettern, M.; Blumenthal-Barby, D.; Eisert, P.


Eisert, P. ; European Association for Computer Graphics -EUROGRAPHICS-; Association for Computing Machinery -ACM-:
6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications, MIRAGE 2013. Proceedings : June 6th - June 7th, 2013, Berlin
New York: ACM, 2013
ISBN: 978-1-4503-2023-8
Art. 2
International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications (MIRAGE) <6, 2013, Berlin>
Fraunhofer HHI ()

We propose a method for creating flexible-viewpoint facial video from monocular input employing highly detailed static 3D reconstructions of an actor's head. We use the term flexible-viewpoint to indicate that the viewpoint can be arbitrarily chosen (as in free-viewpoint video), but from a restricted set of viewing directions. Our method enables dynamic changes of the viewpoint without requiring estimation of the head geometry form the video sequences which is hard with the methods typically used for creating free-viewpoint video. Alongside video capture of a certain actor, we record static high resolution stereo images of the actor's head and face. From these images, we create a detailed 3D model of the head by image-based reconstruction methods. We propose two methods to register this 3D model to the a starting frame of the video stream following the actors head and facial action. Furthermore, we show how model-based tracking over the whole video sequence provides pre cise head pose estimates for each video frame. Once the registration is complete, the 3D model serves as geometric proxy for image-based rendering techniques in order to create novel viewpoints using the video stream as texture.