An Efficient Implementation of the Threshold-based VNG Demosaicing with Reduced Calculations

Color Filter Arrays (CFA) are essential components of digital cameras and image sensors to capture the color information needed to produce full-color images from only a single image sensor per pixel. Many methods and algorithms have been proposed to recover the missing color information of CFAs. In this work, we use a simplified version of the Theshold-based Variable Number of Gradients algorithm proposed by Chang et al. to estimate the full-color information from Bayer images. We also show that the slight modification to algorithm does not effect images quality while making it more compatible with hardware. We propose an efficient implementation of the algorithm that reduces the number of calculations per pixel at the cost of increased memory resources. Our implementation targets an image processing pipeline in an FPGA platform which is short on LUTs and FF resources but has DSPs and BRAMs to spare. We buffer the absolute differences and average color components to be shared and re-used between neighboring pixels, on two levels: within the same row, and between different rows. The latter strategy reduces the number of absolute differences calculated every cycle from 32 to 4 and average color components from 32 to 6. However, the memory requirements are increased from storing 4 image rows to 18 image rows. We implement the solutions on an FPGA using high-level synthesis (HLS) and optimize it to further reduce resources.