Deep learning for occlusion aware RGB-D image completion for structured light measurements

In our work we propose a deep learning solution to complete RGB-D images that are acquired by a NIR structured light scanner with an additional RGB camera that measures the visible spectrum. Building on works on image inpainting, we designed and trained a neural network architecture that takes the available fringe and color images as well as the reliably measured depth information and completes the depth images. We particularly focus on occlusion-caused image artifacts that naturally occur due to geometric visibility constraints. Hence, we are able to reconstruct a dense depth image from the viewpoint of the RGB camera, which can be used for further post-processing and visualization purposes.