Learn from the Best: Harnessing Expert Skill and Knowledge to Teach Unskilled Workers

Experts make complex skills look easy, but learning from experts is not only a matter of observation, but also feedback and reflection. Whereas industrial tasks in the domains of manufacturing and assembly assume a standardized work procedure supported by precision manufactured parts, several other domains where natural products are processed demand a high degree of background knowledge and skill from workers due to high within-product variability. The potential of using assistants in these domains to transfer this expert knowledge to novice workers has rarely been explored. In this paper, we explore how in the rarely studied domain of food-processing, expert know-how in accomplishing a complex task can be analyzed via state-of-the-art machine learning techniques in a multi-modal manner, so that specific features can be detected and tracked to instruct and provide feedback to beginners. We report on the performance and limitations of our approach to activity tracking and discuss its feasibility. A final review with the expert provided additional insights, which we integrated into our approach. We conclude with a summarized framework for capturing and conveying expert knowledge in the industrial domain.