Depth Image Enhancement with Sensor Fusion CycleGAN for Bin Picking Applications

For the task of random bin picking, a sensor that produces high quality 3D measurement data is fundamental. Nowadays, mainly expensive high-end 3D cameras fulfil these requirements and provide high resolution 3D point clouds or depth images with a low level of shadowing effects and noise. At the same time, there is a lot of research on various topics connected to depth image enhancement, like depth image completion or image super resolution. In this paper, we present a novel approach to depth image enhancement for bin picking applications using a Sensor Fusion Cycle Generative Adversarial Network (SF-CycleGAN). Our method combines RGB and depth images to reduce noise and shadowing effects. We train our model using both real and artificial training data, with the artificial data mapped to the domain of a low-cost RGB-D sensor using the CycleGAN approach. We compare our SF-CycleGAN to a U-Net-based approach and demonstrate its effectiveness in improving depth image quality.