Generic and robust video coding with texture analysis and synthesis

A video coding approach that uses texture analysis at the encoder and synthesis at the decoder is presented. It is characterized by scaling of texture reconstruction accuracy at the decoder according to perceptual relevance to the viewer. A closed-loop concept is proposed for a fully automated approach that can be integrated into any video codec. Besides texture analysis and synthesis, the loop encompasses a video quality assessor (VQA) that is roughly three times as complex as PSNR and performs as well as significantly more complex standardized perceptual measures. The VQA operates local and global consistency evaluations, where the outcome is utilized to correct segmentation masks provided by the texture analyzer, such that synthesis results can be improved. The proposed method is generic because rigid and non-rigid textures can be handled. Experimental results are presented that show that significant bit-rate gains can be achieved compared to H.264/AVC without our method.