Eye tracking based interaction with 3D reconstructed objects

This paper presents a method on how users might interact with objects which have been reconstructed from multiple images. Tracking the user's eye movement for HCI in an augmented reality environment provides a convenient, natural, and high-bandwidth source for navigating and zooming in/out of the 3D reconstructed object. The calculated 3D eye position, which is detected by multiple cues from a stereo camera, is synchronized with the 3D position of 3D reconstructed object. Our proposed method of image based 3D modelling is based on voxel carving with photo consistency check. Experimental results show that our proposed interaction methodology using a new solution of 3D eye tracking can be successfully applied for useful tools in context aware applications and improve its usability.