A Multi-camera System for Human Detection and Activity Recognition

The employment of mobile manipulators in the factory to assist human workers is increasing. As the mobile manipulator moves and performs tasks in an unstructured environment with human workers, it should be able to detect the human workers and understand their activity. In this work, a multi-camera system is proposed, consisting of a thermal camera, a stepper motor and a depth camera. The system is able to detect the human workers around the mobile manipulator and to recognize the activity. In addition, the paper describes the comparison of a recognition pipeline based on 2D skeleton key points detected from color images using OpenPose with the state-of-the-art 3D skeleton based models. It shows that 2D data based model in general have better results than 3D data with the same model parameter.