Augmenting image datasets for quality control models using CycleGANs

Deep learning (DL) has proven to be a powerful tool for solving common machine vision tasks, such as image classification, defect segmentation and defect recognition. Usually, training DL models requires significant amounts of annotated data samples, which are generally sparse or of inadequate quality in many quality assurance applications in the engineering domain. Especially the thorough annotation of data yields a major obstacle for the generation of industrial datasets, since it is a complex, time-consuming task requiring expert knowledge of the process under examination. Further, the rareness of defects in rather stable production processes can lead to highly unbalanced datasets, hampering the training process. Combined with the seldom distribution of industrial data due to privacy concerns, the lack of data often hinders the adoption of DL approaches for quality assurance. Recently, network structures following the design of Generative Adversarial Networks (GANs) show astonishing results in the field of image synthesis and neural style transfer. Given a set of unpaired images from two domains, cycle-consistent GANs (CycleGANs) learn how to translate a given image from one domain to the other and vice-versa. This capability can be exploited to augment datasets in a controllable manner in order to alleviate the problems arising in the application of DL for realizing vision-based quality control. This work investigates the employment of CycleGANs to extend the image datasets for two use cases, the detection of pores in computed tomography data and the detection of surface defects on sheared edges of fine blanked parts. Given randomly generated binary masks, the trained CycleGANs are capable of generating an arbitrary amount of synthetic yet realistic images in the desired domains, alleviating the problems of both the data amount and the necessary annotations and demonstrating the great potential of image synthesis using GANs.