Limitations of Anomaly Detection: Beyond which Size Defects can be Reliably Recognized

Anomaly detection is one of the most popular fields for computer vision in industrial applications. The idea of training machine learning only on defect-free objects saves enormous amounts of integration effort. The state of the art shows that current methods on public data sets (e.g. MVTec AD data set) have already solved the problem with AUROC segementations scores of more than 99 %. But how accurate are these methods really? In this paper, one current method from the field of supervised learning and anomaly detection is evaluated on two problems. Each problem contains a defect pattern that grows in 11 steps. This work shows that the defect is already reliably detected from a relative size of 0.03 % of the pixels in the image.