Depth-Aware Dynamic Projection Mapping using High-speed RGB and IR Projectors

In this paper, we propose a strong system for dynamic projection mapping (DPM). It consists of a high-speed 500-fps camera, high-speed 24-bit 947-fps RGB projector, and newly developed 8-bit 2,880-fps infrared (IR) projector. This configuration allows us to capture the depth map and project the depth-aware image at a high frame rate. We also realize 0.4-ms markerless 3D pose tracking using the depth map by leveraging small inter-frame motions under high-frame-rate capture. We exploit these captured data and apply the depth-aware DPM to an entire scene with low latency, in a setting in which tracking-based and modelless mapping are combined and performed simultaneously.