Deep Learning-Based Action Detection for Continuous Quality Control in Interactive Assistance Systems

Interactive assistance systems have shown to be useful in various industrial settings, in particular those involving human labor like manual assembly of workpieces. Current systems support workers based on different technologies like projection-based augmented reality, hand or tool tracking or automated inspections using computer vision techniques. While these technologies help to increase product quality significantly, existing solutions are not able to monitor the entire process, which makes it difficult to detect process errors.
In this paper, we present a deep-learning based approach for continuous on-the-fly quality control within an interactive assistance system. By using labeled video data of an assembly process, a model can be trained that automatically recognizes and distinguishes single actions and thus control the sequence of subsequent work processes. By integrating the system into the interactive assistance systems, users are made aware on any process errors.
Besides presenting the concept and implementation of our deep-learning integration into the assistance system, we describe the created industrial assembly-oriented dataset and present the results from our technical evaluation that shows the potential of applying deep-learning methods into interactive assistance systems.