Intra-Inter Prediction for Versatile Video Coding Using a Residual Convolutional Neural Network

This paper presents a convolutional neural network (CNN) based solution for inter prediction in Versatile Video Coding (VVC). Our approach aims at improving the motion-compensated prediction signal of inter blocks with a residual CNN that incorporates spatial and temporal reference samples, i.e., intra-inter prediction. It is motivated by two considerations. Firstly, incorporating intra reference samples has the potential to improve the prediction signal by adapting it to the reconstructed signal in its immediate vicinity. Secondly, neural network-based methods offer a higher degree of signal adaptivity than conventional signal processing methods. The proposed network is integrated as a post-processing module for inter predicted blocks in VVC. Experimental results show that average bit rate savings of 3.2 % to 4.0 % can be achieved for the random access configuration.