Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Projection Distortion-based Object Tracking in Shader Lamp Scenarios

: Gard, N.; Hilsmann, A.; Eisert, P.


IEEE transactions on visualization and computer graphics 25 (2019), Nr.11, S.3105-3113
ISSN: 1077-2626
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
Bundesministerium fur Wirtschaft und Energie BMWi (Deutschland)
01MD18008B; DigitalTWIN
Fraunhofer HHI ()

Shader lamp systems augment the real environment by projecting new textures on known target geometries. In dynamic scenes, object tracking maintains the illusion if the physical and virtual objects are well aligned. However, traditional trackers based on texture or contour information are often distracted by the projected content and tend to fail. In this paper, we present a model-based tracking strategy, which directly takes advantage from the projected content for pose estimation in a projector-camera system. An iterative pose estimation algorithm captures and exploits visible distortions caused by object movements. In a closed-loop, the corrected pose allows the update of the projection for the subsequent frame. Synthetic frames simulating the projection on the model are rendered and an optical flow-based method minimizes the difference between edges of the rendered and the camera image. Since the thresholds automatically adapt to the synthetic image, a complicated radiometric calibration can be avoided. The pixel-wise linear optimization is designed to be easily implemented on the GPU. Our approach can be combined with a regular contour-based tracker and is transferable to other problems, like the estimation of the extrinsic pose between projector and camera. We evaluate our procedure with real and synthetic images and obtain very precise registration results.