A continual active learning approach to adapt neural networks to distribution shifts in quality monitoring applications

Neural networks show great potential for quality monitoring in manufacturing. However, obtaining suitable and comprehensive training datasets remains a challenge. Moreover, discrepancies between training and inference data distributions can lead to a degradation of model performance. This issue is especially relevant in volatile settings like in high-mix, low-volume production or in remanufacturing, where product variants or observed defect patterns frequently change. This currently hinders the application of machine learning methods in such scenarios. Therefore, we propose a method for ongoing adaptation of machine learning models, i.e., neural networks, during operations. Manual efforts for quality assurance and data annotation are reduced by involving human feedback only when there is a risk of incorrect model predictions and by using that feedback to adapt a model in case of changed data distributions. To accomplish this, the proposed method combines approaches from active and continual learning for targeted sample selection and efficient model adaptation. An extensive experimental evaluation is performed using two application scenarios. We find that a sample selection based on a simple threshold on the model’s confidence score yields a good trade-off between manual effort and the overall system’s classification performance. Additionally, the experiments demonstrate that by warm starting model training and regularizing the training process with a small number of historical samples the necessary training time for model adaptation can be significantly reduced.