Multi 3D-Sensor Based Human-Robot Collaboration with Cloud Solution for Object Handover

This work introduces a human-robot interactive setup incorporating multiple 3D camera sensors and based on a cloud computing infrastructure. A software solution is proposed, utilizing a ROS environment, the PCL library, and the OpenNI tracker for processing RGBD camera data. To ensure safe collaboration and workspace sharing between humans and the UR5e robot, camera data is employed to generate a 3D occupancy map of the robot’s work cell, virtually recreating its environment. The minimal distance between the robot and any obstacles, such as humans, is calculated, allowing for real-time adjustment of the robot’s speed based on this distance. An autonomous object handover process, encompassing both human-to-robot and robot-to-human exchanges, is developed and implemented. Steps such as grasping confirmation and object grasping pose computation are managed using camera data and a skeleton tracker to determine the human hand’s position within the work cell. The work cell is integrated with an industrial cloud solution, the Manufacturing Service Bus (MSB), enabling remote monitoring of the system state and data collection. A variant of the system leveraging remote computing through the MSB is also explored and assessed.