Finding optimal decision boundaries for human intervention in one-class machine-learning models for industrial inspection

Anomaly detection with machine learning in industrial inspection systems for manufactured products relies on labelled data. This raises the question of how the labelling by humans should be conducted. Moreover, such a system will most likely always be imperfect and potentially need a human fall-back mechanism for ambiguous cases. We consider the case where we want to optimise the cost of the combined inspection process done by humans together with a pre-trained algorithm. This gives improved combined performance and increases the knowledge of the performance of the pre-trained model. We focus on so-called one-class classification problems which produce a continuous outlier score. After establishing some initial setup mechanisms ranging from using prior knowledge to calibrated models, we then define some cost model for machine inspection with a possible second inspection of the sample done by a human. Further, we discuss in this cost model how to select two optimal boundaries of the outlier score, where in between these two boundaries human inspection takes place. Finally, we frame this established knowledge into an applicable algorithm and conduct some experiments for the validity of the model.