Robot-aided pointing gesture recognition in intelligent environments

In research field of Intelligent Environments explicit control of contained devices is a major topic. Hand and arm gestures represent an intuitive way to select and control devices and can be gathered by inexpensive depth cameras like the Microsoft Kinect. But for covering multiple rooms several depth cameras are required, resulting in increasing costs and complexity of such a system in larger home areas. To face this problem we use a mobile robot to permanently follow and track the user. The user can simply point from any place at a device in the environment with his arm to control it. Therefore the robot is equipped with a consumer depth camera used to perform gesture recognition and self-localization for computing the global pointing direction. The pointing direction is communicated to the system of the Intelligent Environment which checks for intersection with known devices and changes their state. To prove the feasibility of our approach we realized a prototypical implementation using a real robot and evaluated it with a group of candidates. Finally we compared it with a gesture control system using multiple cameras focusing on user acceptance and costs.