Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Going beyond free viewpoint: creating animatable volumetric video of human performances

: Hilsmann, A.; Fechteler, P.; Morgenstern, W.; Paier, W.; Feldmann, I.; Schreer, O.; Eisert, P.

Volltext ()

IET computer vision 14 (2020), Nr.6, S.350-358
ISSN: 1751-9632
ISSN: 1751-9640
European Commission EC
H2020; 762021; Content4All
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
Zeitschriftenaufsatz, Elektronische Publikation
Fraunhofer HHI ()

An end‐to‐end pipeline for the creation of high‐quality animatable volumetric video of human performances is presented. Going beyond the application of free‐viewpoint video, the authors allow re‐animation and alteration of an actor's performance through the enrichment of the captured data with semantics and animation properties. Hybrid geometry‐ and video‐based animation methods are applied that allow a direct animation of the high‐quality data itself instead of creating a CG model that resembles the captured data. Semantic enrichment and animation are achieved by establishing temporal consistency followed by automatic rigging of each 3D frame using a parametric human body model. The hybrid approach combines the flexibility of classical CG animation with the realism of real captured data. For the face, coarse movements are modelled in the geometry only, while very fine and subtle details, often lacking in purely geometric methods, are captured in video textures, which can interactively be combined to form new facial expressions. On top of that, regions that are challenging to synthesise, such as the teeth or the eyes, are learned and filled in realistically in an autoencoder‐based approach. This study covers the full pipeline from capturing, volumetric video production, and enrichment with semantics for the final hybrid animation.