A flexible approach to gesture recognition and interaction in X3D

With the appearance of natural interaction devices such as the Microsoft Kinect or Asus Xtion PRO cameras, a whole new range of interaction modes have been opened up to developers. Tracking frameworks can make use of the additional depth image or skeleton tracking capabilities to recognize gestures. A popular example of one such implementation is the NITE framework from PrimeSense, which enables fine grained gesture recognition. However, recognized gestures come with additional information such as velocity, angle or accuracy, which are not encapsulated in a standardized format and therefore cannot be integrated into X3D in a meaningful way. In this paper, we propose a flexible way to inject gesture based meta data into X3D applications to enable fine grained interaction. We also discuss how to recognize these gestures if the underlying framework provides no mechanism to do so.