Intuitive control of tele-operated inspection and maintenance robots through automatic object detection and AR visualization

This paper presents a method to assist operators of mobile robots applied for inspection and maintenance in industrial process plants. The proposed method enables the automatic detection of circular objects - specifically circular hand wheels - by the 3D-sensors of the robot. The detected objects are illustrated on the graphical user interface by augmenting the camera image of the robot with the detected objects. To compensate inaccuracies, the user can modify the properties of each detected object manually on the screen. Finally he can specify the desired grasping position for the robot manipulator on one of the detected wheels. Based on the acquired environment information the robot will then be able to grasp this wheel automatically and without collisions with the environment. By controlling the applied forces on the gripper and by using the parameters of the wheel specified previously, the robot is further able to assist the user to rotate the wheel by for example pushing a joystick in the desired rotation direction. Current work focuses on optimizing and integrating the single components.