RNNSC: Recurrent Neural Network-Based Stereo Compression Using Image and State Warping

Stereo images are used in various applications, such as autonomous driving, surveillance, robotics, and 3D-TV. Those images are captured by two horizontally adjacent cameras, capturing a scene from two different points of view. In this work, we propose an end-to-end trainable recurrent neural network (RNN) for stereo image compression, which we call RNNSC. The RNN allows variable compression rates without retraining of the network due to the iterative nature of the recurrent units. The proposed method makes use of the redundancies, to reduce the overall bit rate. Each image in the stereo pair has its separate encoder and decoder network similar to [1]. We propose to share the mutual information between the stereo pair networks by warping the hidden states of one codec network to the other with the help of disparity information that is coded and transmitted independently via JPEG2000. Moreover, we also improve the quality of the shared mutual information by eliminating wrong information by estimating and applying occlusion maps which are computed with a convolutional neural network without direct supervision. The proposed method outperforms all tested image codecs on MS-SSIM, a perceptual metric capturing the structural quality of an image, as shown in Table 1.